ARI  Research  Note  95-44 


Battalion  -  Battle 
Staff  Training  System 

Charles  R.  Andre 

BDM  Federal,  Inc. 


Margaret  S.  Salter 

U.S.  Army  Research  Institute 


OTIC 

ELECTE 

SEP  2  8  1995 


infantry  Forces  Research  Unit 
Thomas  J.  Thompson,  Acting  Chief 

Personnei  and  Training  Systems  Research  Division 
Zita  M.  Simutis,  Director 


August  1995 


19950925  155 

United  States  Army 

Research  Institute  for  the  Behavioral  and  Social  Sciences 

Approved  for  public  release;  distribution  is  unlimited. 


DTIC  QUALITY  BiePUCTED  8 


U.S.  ARMY  RESEARCH  INSTITUTE 

FOR  THE  BEHAVIORAL  AND  SOCIAL  SCIENCES 


A  Field  Operating  Agency  Under  the  Jurisdiction 
of  the  Deputy  Chief  of  Stalf  for  Personnel 


Edgar  M.  Johnson 
Director 


Research  accomplished  under  contract 
for  the  Department  of  the  Army 

BDM  Federal,  Inc. 

Technical  review  by 
Robert  J.  Pleban 


Accesion  For 


NTIS  CRA&l 
OTIC  TAB 

Unannounced 

Justification 


□ 


By _ 

Distribution/' 


Availability  Codes 

Dist 

Avail  and/or 
Special 

1 

NOTICES 

DISTRIBUTION:  This  report  has  been  cleared  for  release  to  the  Defense  Technical  Information 
Center  (DTIC)  to  comply  with  r^ulatoiy  requirements.  It  has  been  given  no  primary  distribution 
other  than  to  DTIC  and  will  be  available  only  through  DTIC  or  the  National  Technical 
Information  Savice  (NTIS). 

FINAL  DISPOSITION;  This  report  may  be  destroyed  what  it  is  no  longa  needed.  Please  do  not 
return  it  to  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

NOTE:  The  views,  opinions,  and  findings  in  this  rqtort  are  those  of  the  aulhor(s)  and  should  not 
be  construed  as  an  official  Department  of  the  Army  position,  policy,  or  decision,  unless  so 
designated  by  offia  authorized  documoits. 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
0MB  No.  0704-0188 


PUiber^MrtinghrdnforMicoaadjanorinfarnMticnittstinutedtomng*  1  hcurpo* response,  inc^ldtl^th•timt  for rteiewinginitnicUoas.senhiiij existing <M« sources.  gelhain9«i<lmairUinm(tbed««ne«<M,a>dconip1etingndr«*icirin(theM^^  Send cotnmets  ragsrding this bwden 

ertsnteor  «vdher  ig»ct  of  Ihii  colledion  of  infonn^on,  mctuding  suggestions  for  reAjcing  this  bvtlen  to  Wsshington  He*lqu*t«rs  Sereices,  Director^  for  informstion  OporStioos  sod  Reposts,  1215  Jefferson Dteis  Highwiy,  Suite  1204,  Arlington,  VA  22202-4302,  md  to  the  Office  of  Mmgemeri  ml  Budget, 
Paperwork  Reduction  Project  (0704-0180),  Wishington,  DC  20503 


1.  AGENCY  USE  ONLY  (Leave  Blank) 


2.  REPORT  DATE  1995,  AugUSt  3.  REPORT  TYPE  AND  DATES  COVERED  FINAL  6/93  -  1/95 


4.  TITLE  AND  SUBTITLE 

Battalion-Battle  Staff  Training  System 


6.  AUTHOR(S) 

Charles  Andre  (BDM)  and  Margaret  S.  Salter  (ARI) 


5.  FUNDING  NUMBERS 

MDA903-92-D-0075-0011 

623007A 

A794 

2126 

C04 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

BDM  Federal,  Inc. 

1801  Randolph  St.,NE 
Albuquerque,  NM  87106 


8.  PERFORMING  ORGANIZATION  REPORT  NUMBER 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
ATTN:  PERI-IJ 
5001  Eisenhower  Ave. 

Alexandria,  VA  22333-5600 


10.  SPONSORING/MONITORING  AGENCY  REPORT  NUMBER 


ARI  Research  Note  95-44 


11.  SUPPLEMENTARY  NOTES  ' 

COR:  Thomas  J.  Thompson 


12a.  DlSTRIBUnON/AVAILABILITY  STATEMENT 


Approved  for  public  release;  distribution  is  unlimited. 


13.  ABSTRACT  (Maximum  200  words): 

This  report  documents  the  design  and  development  of  13  courses  of  instruction  for  the  Battalion-Battle  Staff  Training  system 
(BN-BSTS).  BSTS  is  a  set  of  training  materials  for  battalion-level  staff  officers,  a  mixture  of  text  and  CD-ROM  computer-based 
instruction  (CBI).  Designed  for  stand  alone  or  local  area  network  linked  training  systems,  the  BN-BSTS  was  developed  for  use  by 
the  U.S.  Army  National  Guard  (ARNG).  Part  of  the  training  challenge  for  ARNG  combat  arms  staff  members  is  due  to  conflicts 
with  other  duties.  To  help  alleviate  this  problem,  distributed,  multimedia  (paper-based  and  computer  based),  individualized 
instruction  for  battalion  staff  personnel  was  developed.  The  resulting  BSTS  project  provides  a  prototype  staff  officer  training 
program  with  courses  which  cover  individual  battalion  staff  functional  areas  and  those  individual  tasks  required  to  prepare  the  battle 
staff  members  for  collective  battle  staff  tasks.  This  project,  sponsored  under  the  Advanced  Research  Projects  Agency  (ARP  A) 
program  umbrella  of  Simulation  in  Training  for  Advanced  Readisness  (SIMITAR),  is  coordinated  with  two  other  programs: 
Simulation-Based  Multiechelon  Training  for  Armor  Units  (SIMUTA)  and  Combat  Service  Support  (CSS)  Training  System 
Development  for  the  National  Guard. 


14.  SUBJECT  TERMS 


Battalion  Battle  Staff  Training  System  (BN-BSTS)  U.  S.  Army  National  Guard  (ARNG) 

Staff  Officer  Functional  Area  Training  Readiness  Research  Projects  Agency  (ARP A) 

Simulation  in  Training  for  Advanced  Readiness  (SIMITAR)  (Cont.) 


17.  SECURITY  CLASSIFICATION  OF  REPORT 


18.  SECURITY  CLASSIFICATION  OF 
THIS  PAGE 


19.  SECURITY  CLASSIFICATION  OF 
ABSTRACT 


Unclassified 


Unclassified 


Unclassified 


15.  NUMBER  OF 


PAGES  107 


20.  LIMITATION  OF  ABSTRACT 


Unlimited 


Prescribed  by  ANSI  Std  239 


ARI  Research  Note  95-44 


14.  SUBJECT  TERMS  (Continued) 

Simulation-Based  Multiechelon  Training 
for  Armor  Units  (SIMUTA) 
Computer-based  instruction  (CBI) 


11 


ACKNOWLEDGMENTS 


I  am  grateful  for  the  incisive  research  conducted  by  the 
Infantry  Forces  Research  Unit/  Army  Research  Institute/  Fort 
Benning/  Georgia/  in  the  creation  of  the  first  Commander's  Battle 
Staff  Handbook  upon  which  this  program  was  based.  A  special 
thanks  to  the  Acting  chief/  Dr.  Thomas  J.  Thompson/  for  his  sound 
counsel/  advice/  and  support  during  the  production  of  the 
prototype  training  program. 

Thanks  also  to  the  Operations  group/  National  Training 
Center  (especially  the  Scorpion  Team)  /  for  their  advice/ 
assistance/  and  contributions  to  the  development  of  the 
battlestaff  training  program/  particularly  for  their  lessons 
learned. 

I  wish  also  to  thank  the  following  individuals  for  their 
detailed  research  and  expertise  that  produced  the  program: 
designers  and  developers  Ms.  Deborah  Sutton,  Mr.  Jim  Centric,  Mr. 
William  S.  Godwin,  Dr.  Sherry  Frese,  Mr.  Stephen  C.  Livingston, 
Mr.  John  M.  Kraemer,  Ms.  Linh  Nguyen,  Mr.  Larry  Heard,  Ms. 
Meredith  Taylor,  Dr.  Nancy  Atwood,  Ms.  Judith  Zittrauer,  Ms.  Lori 
Byrd-Spring,  and  Mr.  Jack  Briscoe.  For  outstanding 
administrative  support,  thanks  to  Ms.  Jennifer  D.  Kasprzyk. 


iii 


BATTALION-BATTLE  STAFF  TRAINING  SYSTEM 


CONTENTS _ _ _ 

,  Pa^ 

INTRODUCTION . 1 

Background . * . 1 

Prior  Research . .2 

METHOD  .  3 

Instructional  Overview . 4 

Instructional  Concept  .  5 

Instructional  Content  .  5 

Training  Management  .  7 

Authoring  of  Courseware  .  8 

Realism  in  Training . 10 

Quality  Assurance . 10 

RESULTS . 10 

Testing  and  Evaluation . 11 

CONCLUSIONS . 13 

REFERENCES . 15 

APPENDIX  A.  Hardware  and  Software  Configuration  .  A-1 

B.  Introduction  to  BDM' s  Environment  for  Multimedia 

Interactive  Instruction  .  B-1 

C.  Standards  Used  for  the  Development  of  ARNG 

Computer  Based  Instruction  .  C-1 

D.  BETA  Test  Results . D-1 

LIST  OF  TABLES 

Table  1.  Estimated  Number  of  Course  Hours . 11 

LIST  OF  FIGURES 

Figure  1 .  Model  Course  Map  .  4 


V 


CONTENTS  (Continued) 


Page 


Figure  2.  Course  Design  and  Development  Process  Flow 

3.  TMS  Hardware  Architecture  . 

4.  CBI  Flow  Chart  . . . 

5.  Beta  Testing  Process . 12 


vi 


^  00 


Battalion-Battle  Staff  Training  System 


Introduction 

The  purpose  of  this  report  is  to  document  the  design  and 
development  of  thirteen  courses  of  instruction  delivered  for  the 
Battalion-Battle  Staff  Training  System  (BN-BSTS) .  BN-BSTS  is  a 
set  of  training  materials  for  battalion  level  staff  officers,  a 
mixture  of  text  and  computer  based  instruction  (CBI) .  The 
package  includes  CD-ROM  based  programs,  with  instructional  and 
training  management  systems .  Designed  for  stand  alone  or  local 
area  network  linked  training  systems,  the  BN-BSTS  was  developed 
for  use  by  the  U.S.  Army  National  Guard  (ARNG) .  The  work  was 
performed  by  BDM,  Federal,  Inc.,  under  contract  to  the  Army 
Research  Institute  (ARI)  and  the  Advanced  Research  Projects 
Agency  (ARPA)  (MDA903-92-D-0075,  D0#0011) .  BDM  provided 
management,  authoring,  and  military  subject  matter  expertise. 

The  work  was  performed  under  the  direction  of  the  ARI  Infantry 
Forces  Research  Unit  (IFRU)  at  Fort  Banning,  Georgia  from  May 
1993  through  January  1995. 

Background 

The  ARPA  Land  Systems  Office  and  the  ARNG  Advanced 
Distributed  Simulation  Program  have  focused  on  the  unique 
training  needs  of  the  Reserve  Components .  Part  of  the  training 
challenge  for  ARNG  combat  arms  staff  members  is  due  to  conflicts 
with  other  required  multiple  unit  training  assembly  (MUTA) 
battalion  duties.  To  help  alleviate  this  problem,  the  ARPA, 

ARNG,  and  ARI  initiated  the  development  of  distributed,  multi- 
media  (paper-based  and  computer-based)  individualized  instruction 
for  battalion  staff  personnel.  The  resulting  BSTS  project 
provides  a  prototype  training  program  with  courses  which  cover 
individual  battalion  staff  functional  areas  and  those  individual 
tasks  required  to  prepare  the  battle  staff  members  for  collective 
battle  staff  tasks. 

This  project,  sponsored  under  the  ARPA  program  umbrella  of 
Simulation  in  Training  for  Advanced  Readiness  (SIMITAR) ,  is 
coordinated  with  two  other  programs:  Simulation-Based 
Multiechelon  Training  for  Armor  Units  (SIMUTA)  and  Combat  Service 
Support  (CSS)  Training  System  Development  for  the  National  Guard. 
BN-BSTS  focuses  on  the  development  of  a  prototype  staff  officer 
training  program  available  to  ARNG  officers  for  completion  at 
home  or  in  the  armory.  This  program  applies  innovative  research, 
application,  and  technologies  of  instructional  design,  computer 
science,  and  multi-media  training  strategies  to  enhance  learning. 
The  BN-BSTS  will  be  distributed  to  two  ARNG  test  brigades 
selected  for  participation  in  a  1996-1997  evaluation. 
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Prior  Research 


Battle  staff  training.  The  need  for  formal  battle  staff 
training,  especially  for  units  conducting  premobilization 
training  or  training  for  a  rotation  to  a  Combat  Training  Center 
(CTC)  became  evident  from  ARI's  research  on  home  station 
determinants  of  effective  unit  performance  at  the  CTCs  (see  Holz, 
Hiller  &  McFann,  1994) .  The  study  of  units  at  the  National 
Training  Center  (NTC)  and  the  Joint  Readiness  Training  Center 
(JRTC)  showed  that  special  training  focused  on  the  battle  staff 
(especially  orders  drills)  led  to  better  unit  performance. 

Research  on  light  infantry  units  found  that  staff  officer 
training  in  the  appropriate  functional  area  was  not  provided 
prior  to  assignment  to  maneuver  battalion  or  brigade  staff 
(Thompson,  Thompson,  Pleban  &  Valentine,  1991) .  The  only  officer 
training  available  was  supervised  by  the  commander,  on-the-job. 

As  a  result,  the  battle  staff  did  not  recognize  deficiencies  in 
synchronization  of  critical  command  and  staff  technical  and 
tactical  activities  until  after  a  rotation  was  completed. 

The  battle  staff  competency  issue  has  been  addressed  and 
research  has  been  conducted  on  the  problem  of  training  and 
synchronization  of  battle  staffs.  An  interim,  solution.  The 
Commander' s  Battle  Staff  Handbook  (Pleban,  Thompson,  & 

Valentine,  1993)  was  developed,  and  has  been  distributed  to  both 
Active  Component  (AC)  and  Reserve  Component  (RC)  personnel.  The 
handbook  provides  descriptions  of  individual  battle  staff  duties, 
responsibilities,  and  references  for  key  staff /slice  officers  and 
the  command  sergeant  major.  It  was  developed  from  staff  material 
provided  by  TRADOC  branch  schools,  interviews  with  subject  matter 
experts,  and  from  the  assessment  of  unit  performance  by  observers 
at  the  CTCs .  The  Commander's  Battle  Staff  Handbook  was  the 
primary  base  document  for  the  BN-BSTS  courses . 

Battle  Staff  Integration.  BN-BSTS  also  incorporated 
research  on  the  concept  of  organizational  competence  and  seven 
organizational  processes  "required  by  the  organizational  system 
(the  battle  staff)  for  effective  accomplishment  of  missions" 
(Olmstead,  1992,  p.  V-1) .  Olmstead's  tenets  became  training 
objectives  in  the  common  core  course  of  instruction  and  were 
integrated  into  the  remaining  courses .  This  instruction  will 
ensure  that  individual  officers  and  the  entire  battle  staff 
possess  the  requisite  functional  and  team  competency  skills  to 
fully  synchronize  individual  efforts  into  the  collective  staff 
effort.  This  process  will  lead  to  optimal  performance  during  the 
planning,  preparation  and,  to  a  lesser  extent,  execution  phases 
of  the  given  mission  or  individual/collective  tasks. 
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Asynchronous  training.  The  BN-BSTS  incorporated  aspects  of 
SMART  (System  for  Managing  Asynchronous  Remote  Training)  which 
demonstrated  a  methodology  for  the  delivery  of  individualized 
instruction  to  ARNG  officers  widely  distributed  throughout  the 
country  (Hahn,  et  al . ,  1990,  1991).  SMART  allowed  students  to 
interact  on  their  own  time,  asynchronously,  with  fellow  students 
and  instructors  via  telephone  links,  similar  to  electronic 
bulletin  boards. 


Method 

Text  based  materials  and  storyboards  were  developed  at  Fort 
Banning,  GA.  BDM's  Albuquerque,  NM,  personnel  provided 
assistance  with  CBI,  the  training  management  system  and  the 
performance  measurement  and  feedback  system. 

The  primary  developers  of  the  instructional  materials  for 
the  BN-BSTS  effort  came  from  a  team  of  subject  matter  experts 
(SMEs)  with  extensive  experience  serving  on  AC  and  RC  battalion 
battle  staffs,  and  in  combat  and  at  the  CTCs .  They  determined 
the  individual  training  objectives  for  each  course,  using  as 
reference  the  Commander's  Battle  Staff  Handbook  and  doctrinal 
materials.  They  surveyed  existing  formal  courses.  Army  doctrinal 
publications  (Field  Manuals,  Mission  Training  Plans,  Technical 
Manuals,  Programs  of  Instruction,  etc.),  and  related  materials. 
They  used  information  from  the  Center  for  Army  Lessons  Learned 
(CALL)  extracts,  trends,  reports,  data,  and  observations  from 
past  and  on-going  combat  operations,  and  CTC  After  Action  Reviews 
(AARs)  . 

Programs  of  instruction  were  written,  following  accepted 
instructional  design  models,  plus  development  of  new  models  for 
the  design,  development  and  delivery  of  text-based  and  CBI. 
Included  were  state-of-the-art  hardware  and  software  from  the 
computer  industry,  CBI  software  development  firms,  and  the 
entertainment  industry.  Instructional  development  used  story 
boards . 

Presentation  media  considered  both  the  cost  and  training 
effectiveness  of  existing  materials  and  materials  to  be  designed. 
Maintaining  student  interest  was  essential  because  students  would 
be  required  to  learn  on  their  own.  CBI  presentation  supplemented 
with  text -based  and  companion  doctrinal  publications  provided  an 
inherently  more  interesting  instructional  program  than  existing 
text-based  materials  alone. 

The  CBI  format  was  designed  to  implement  the  diagnostic, 
assessment,  and  performance  measurement  component  of  the  system 
as  well  as  provide  the  primary  instructional  medium  to  provide 
the  battle  staff  officer  with  exercises  and  tutorials  to  apply 
the  instruction  presented. 
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Instructional  Overview  ‘  "i.  .  .. 

CBI  is  the  foundation  for  this  training  systemV  Staff 
officers  receive  their  CBI  materials  through  CD  ROM  in 
conjunction  with  the  accompanying  text  based  materials. 

Technical  specifications  for  the  student  computer  stations  and 
the  manager  station  are  provided  at  Appendix  A. 

The  individual  has  access,  if  the  unit  chooses,  to  other 
staff  members  through  the  use  of  either  an  asynchronous 
electronic  bulletin  board  or  a  synchronous  "chat"  capability. 

They  can  conduct  synchronous  or  asynchronous  outside  class 
meetings  to  help  one  another  with  problems  and  discuss  course 
content.  These  communication  methods  can  prepare  battle  staff 
officers  for  collective  tasks.  (Two  Shiva  net  modems  provide  the 
synchronous  "chat"  capability  for  the  system.) 

The  thirteen  courses  of  instruction  are  composed  of 
subjects;  subjects  are  groups  of  lessons,  and  lessons  are 
composed  of  topics  that  cover  a  major  learning  objective  (see 
Figure  1,  Model  Course  Map) . 
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Lessons  take  one  or  two  hours  to  complete.  The  Training 
Management  System  (TMS)  allows  the  commander,  executive  officer, 
training  officer,  or  other  designated  individual  to  schedule 
courses,  monitor  the  progress  of  the  individual  battle  staff 
officers,  and  manage  the  authoring  of  revisions  or  future  courses 
of  instruction.  (Appendix  B  provides  background  information  on 
the  TMS.) 

CBI  development  used  commercial,  off-the-shelf,  software 
that  met  the  criteria  for  selection  of  CBI  authoring  software. 
Additional  tools  and  templates  were  also  developed  for  use  in  the 
BN-BSTS.  The  templates  are  included  in  the  standards  of 
instruction  emplaced  for  the  program.  Standards  include  learning 
strategies  for  mastery  of  skills  and  knowledge,  techniques  for 
question  and  answer  sessions,  testing  criteria,  screen  design, 
navigation  through  the  courses  of  instruction,  helps,  job  aids, 
and  screen  layout .  (Appendix  C  contains  information  on  the 
Instructional  Standards.) 

Instructional  Concept 

The  multi-media  CBI  and  text  materials  included  graphics, 
still  photographs,  audio,  text,  and  full  motion  video  with  audio. 
Specific  application  of  media  was  based  on  selection  of  the  most 
effective  media  for  teaching  the  learning  objectives.  CBI  was 
used  where  it  is  a  clearly  superior  strategy  of  instructional 
delivery;  especially,  to  present  high  order  cognitive  tasks, 
complex  tasks  that  require  coordination  or  synchronization  with 
other  tasks,  and  practical  application  of  the  skill  or  knowledge 
being  presented.  All  diagnostics,  assessment,  measurement  and 
feedback  were  included  in  CBI  to  provide  student  progress  and 
status  reports  to  the  trainer.  The  instruction  focuses  on 
mastery  of  basic  skills  and  knowledge  followed  by  application  of 
critical  tasks  to  ensure  the  student  can  apply  the  task  in  a 
realistic  environment. 

While  the  instructional  concept  places  the  student  in 
control  of  the  pace  and  sequence  of  instruction,  feedback 
provided  to  the  student  informs  him  or  her  whether  or  not  the 
task  has  been  mastered,  thereby  enabling  him  or  her  to  receive 
remedial  training  if  the  chain  of  command  so  chooses . 

Instructional  Content 


Courses  of  instruction  were  designed  for  the  battalion 
battle  staff,  which  consists  of  the  commander's  primary  and 
special  staff  as  well  as  the  Combat  Support  (CS)  and  Combat 
Service  Support  (CSS)  slices  that  habitually  work  with  the  staff. 
As  a  result  of  the  program  front-end  analysis  (FEA) ,  three 
additional  courses  of  instruction  were  added  to  ensure  the 
delivery  of  a  complete  program  of  instruction  for  the  battle 
staff.  A  needs  assessment,  conducted  to  determine  the  tasks  to 
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be  trained  for  the  selected  battle  staff  officers.,  was  conducted 
for  each  course  and  published  in  the  form  of  the  battalion  battle 
staff  task  list. 

One  SME  was  assigned  primary  responsibility  for  the  design 
and  development  of  each  course  of  instruction.  The  SME,  in 
conjunction  with  an  instructional  designer,  applied  a  functional 
job  analysis  to  develop  the  training  objectives  and  corresponding 
performance  measures  to  be  trained.  (Figure  2  shows  this 
process . ) 


Figure  2.  Course  design  and  development  process  flow. 


SMEs  surveyed  and  documented  all  relevant  doctrinal  and 
training  literature,  lessons  learned,  and  applicable  Army  formal 
courses  and  instructional  materials.  Following  the  literature 
review,  the  SME  and  instructional  designer  applied  the 
design/development  model,  culminating  in  a  final  version  of  the 
text-based  instruction  with  adjunctive  materials  and  tests. 
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Quality  assurance  of  materials  was  accomplished  both  by  peers, 
for  content,  and  an  instructional  designer  for  application  of 
sound  instructional  principles. 

Each  of  the  courses  has  a  listing  of  associated, 
professional  reference  materials  needed  to  complete  the  course  of 
instruction.  Thus,  at  the  completion  of  the  course  the  staff 
officer  will  be  technically  prepared  to  perform  the  required 
staff  function,  and  will  also  know  to  refer,  when  required,  to 
doctrinal  publications  for  specific  application  of  problem 
solving  skills  within  the  functional  area,  and  as  part  of  the 
battle  staff. 

The  most  complex  cognitive  tasks,  and  those  that  were 
determined  to  require  integration  and  synchronization  with  other 
tasks  were  designated  for  application  of  CBI.  Storyboards,  with 
attached  media,  were  provided  to  the  CBI  authors  for  development 
into  CBI  with  specified  training  application,  sequence, 
branching,  and  selected  media. 

Training  Management 

Management  of  the  training  applied  a  modification  of  the 
Training  Management  System  (TMS)  developed  for  use  with  the  BN- 
BSTS  and  CSS  programs  and  designed  specifically  to  enhance  the 
instructor- independent  nature  of  the  distance  learning  used  in 
this  program  (see  Appendix  B) .  The  TMS  is  an  enabling 
environment  for  multi-media  computer-based  training  development 
and  delivery.  The  TMS  integrates  a  system  that  allows  trainers 
and  administrators  to  schedule  students  for  courses,  monitor 
student  progress,  receive  progress  reports,  print  results, 
analyze  measurement  data  for  course  enhancement,  and  modify 
courseware  while  students  are  training  on  a  previous  version  of 
the  course. 

The  Windows  for  Workgroups  (™)  environment  provides  a 
familiar  look  and  feel  across  the  training  media.  A  model  of  the 
hardware  architecture  is  at  Figure  3.  The  TMS  uses  Microsoft's 
Access  (™)  database  to  manage  the  student's  progression  through 
the  course  of  instruction.  Course  templates  standardize  the 
recording  of  student  progress  within  lessons  for  posting  to  the 
data-base.  This  allows  the  commander,  executive  officer, 
training  NCO  or  other  designated  individual  to  track  the  overall 
progress  of  the  staff. 
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Manager  PC 


Integrated  Networking 
Conferencing  PC 


Instructor  Station 


Student  Stations 


At  Home: 
Student  Station(s) 


FigurG  3.  TMS  hardwarG  architGcturG. 


Authoring  of  Courseware 

At  the  armory,  or  at  home,  CBI  is  the  foundation  for  this 
training  system.  Staff  officers  receive  their  instructional 
materials  through  CD  ROM  in  conjunction  with  the  accompanying 
text  based  materials. 

CBI  development  required  three  skills:  subject  matter 
expertise,  knowledge  of  instructional  design,  and  computer 
literacy  skills.  The  program  used  SMEs  that  possessed  both 
instructional  design  and  CBI  design  skills  to  prepare  storyboards 
for  coding  into  CBI .  Authoring  systems  were  available  at  both 
Fort  Banning  and  Albuquerque  to  ensure  quality  assurance 
procedures  could  be  applied  regardless  of  where  the  CBI  was 
coded.  Icon  Author  5.0  (™)  was  used  to  develop  the  CBI, 
templates,  and  standards.  It  has  previously  been  accepted  as 
meeting  the  selection  criteria  for  an  authoring  system  (Park  & 
Seidel,  1989)  and  ensures  compatibility  with  CSS  courses  of 
instruction  being  developed  for  another  related  project  under 
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ARPA  auspices.  The  Icon  Author  package  proved  relatively  easy  to 
learn,  formal  training  was  readily  available,  and  an  experience 
base  on  its  use  resided  within  BDM. 

Most  all  of  the  templates  and  Smart  Objects  developed  for 
use  were  developed  specifically  to  implement  the  design  standards 
of  the  BN-BSTS  program.  Additionally,  templates  and  Smart 
Objects  were  developed  for  the  diagnostic,  performance 
measurement  and  assessment  component  of  the  system  to  enhance  the 
overall  learning.  Figure  4  provides  a  diagram  of  the  CBI  flow 
chart . 


Figure  4.  CBI  flow  chart. 

I 
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Realism  in  Training 


Realism  in  training  was  created  and  maintained  through  the 
use  of  the  common  missions  derived  from  NTC  rotations  and 
developed  and  used  in  the  BN-BSTS,  CSS,  and  SIMUTA  programs. 

These  scenarios,  with  accompanying  orders,  annexes  and  graphics, 
were  applied  to  the  program  to  create  a  realistic  context  for 
training  and  permit  the  use  of  a  tested  "a  way"  solution  which 
allows  the  student  to  review  and  evaluate  the  developed 
solutions . 

The  multi-media  strategy  is  particularly  valuable  for 
providing  realism  and  retaining  interest  in  the  training.  This 
strategy  includes  the  application  of  tactical  scenarios  which  set 
the  context  of  the  instruction  and  practical  exercises.  The 
scenario (s)  were  built  on  the  scenarios  developed  and  used  in 
SIMUTA  programs  (Brown,  1992) .  Students  are  led  through  the 
training  context  (scenario)  applying  their  learned  skills  to 
situations  presented.  Then,  as  appropriate,  they  are  provided  "a 
way"  solutions  against  which  to  compare  their  results. 

Quality  Assurance 

Quality  assurance  (QA)  was  integrated  throughout  the 
developmental  process.  QA  for  BN-BSTS  was  accomplished  by  SMEs 
for  peer  review  of  content,  the  instructional  designer  for 
standards  and  instructional  integrity,  the  program  manager  for 
content  and  standards,  and  computer  programmers  for  functionality 
of  the  code;  thereby  ensuring  quality  instruction  that  attained 
the  training  objective.  QA  occurred  throughout  the  process, 
during  development  of  training  objectives,  performance  measures, 
text,  tests  and  storyboards,  and  the  CBI  process. 

Results 

Thirteen  courses  were  designed  for  the  BN-BSTS  program.  The 
courses  delivered  were  Common  Core  (material  common  to  all 
positions) ,  and  the  battalion  level  Executive  Officer  (XO) ,  SI 
(personnel) ,  S2  (intelligence) ,  S3  and  S3  Air  (operations) ,  S4 
(logistics) ,  Chaplain,  Fire  Support  Officer  (FSO) ,  Engineer,  Air 
Defense  Artillery  (ADA) ,  and  the  Signal  (SIGO)  and  Chemical 
(CHEMO)  officers.  The  estimated  numbers  of  course  hours  are 
shown  in  Table  1.  Totals  for  CBI  hours  are  more  nearly  fixed; 
the  text  hours  may  vary  with  the  reading  speed  and  comprehension 
of  the  student . 


10 


Table  1 

Estimated  Number  of  Course  Hours 


COURSE 

TEXT 

CBI 

TOTAL 

Common  Core 

29.15 

2.00 

31.15 

XO 

18.25 

7.50 

25.75 

SI 

22.00 

7.50 

29.50 

Chaplain 

5.50 

0.75 

6.25 

S2 

28.25 

9.50 

37.75 

S3 

12.05 

4.25 

16.30 

S3  Air 

51.00 

1.75 

52.75 

S4 

10.50 

4.50 

15.00 

FSO 

56.90 

6 . 05 

62.95 

Engineer 

22.50 

2.50 

25 . 00 

ADA 

17.20 

1.75 

18.95 

SIGO 

15.60 

2.00 

17.60 

CHEMO 

43.75 

2.25 

46 . 00 

TOTALS 

332.65 

52.30 

384 . 95 

Testing  and  Evaluation 

The  BN-BSTS  was  tested  through  a  formative  evaluation  were 
process  that  included  Alpha  testing  by  6  to  10  persons  who  are 
SMEs  in  content,  computer  programming,  and/or  instructional 
design.  They  looked  at  the  text -based  materials,  CBI,  and  the 
complete  course  of  instruction  for  instructional  soundness  and 
functionality  of  the  computer  programming.  Figure  5  details  the 
steps  in  the  process;  full  results  are  at  Appendix  D. 

Alpha  testing  of  the  text  was  conducted  by  3  to  5  SMEs  with 
expertise  in  the  specific  battle  staff  position  being  trained. 

The  text  was  reviewed  by  SMEs  from  Fort  Benning  and  other 
proponents  around  the  TRADOC.  Testing  of  CBI  was  conducted  by 
computer  programmers  at  both  Fort  Benning,  GA  and  Albuquerque,  NM 
to  ensure  the  proper  operation  of  all  codes. 

Beta  testing  of  the  complete  courses  of  instruction  by 
representatives  of  the  target  population  was  completed  prior  to 
finalization  of  the  instructional  program.  A  two-week  evaluation 
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at  the  Military  Intelligence  School  verified  application  of  the 
S2  package  to  the  AC.  Test  results  included  a  final  evaluation 
of  the  courses  of  instruction  by  the  officers.  Of  the  12 
objective  questions  asked  of  every  test  officer  on  the  post 
course  survey,  81%  provided  positive  comments,  and  8%  were 
neutral,  and  11%  were  negative. 

Test  officers  were  asked  to  comment  on  each  lesson  (broken 
down  into  text,  CBI,  and  examinations) ,  each  subject,  and  the 
student  guide  as  well  as  provide  overall  comments  and  suggestions 
on  the  course (s)  of  instruction.  The  most  prevalent  comment  was 
for  more  CBI  (all  respondents)  and  others  asked  for  more 
practical  exercises  (PEs)  in  CBI  ("that's  where  we  learn").  Some 
commented  that  historical  references  and  lessons  learned  were  of 
great  value,  as  were  the  graphics  and  humor.  There  was  praise 
for  the  quiz  format,  and  CBI  in  general. 

BETA  testing  of  the  final  course  of  instruction  was 
conducted  on  the  actual  hardware  integrated  into  the  network,  by 
ARNG  battle  staff  soldiers.  To  ensure  accomplishment  of  program 
objectives,  test  results  and  appropriate  recommendations,  were 
included  in  the  final  course  materials  prior  to  final  printing  of 
the  text  or  cutting  of  the  CD  ROM  for  CBI .  BETA  test  results 
were  provided  as  each  course  completed  testing  (see  Appendix  D) . 


Conclusions 

The  Battalion-Battle  Staff  Training  System  has  been 
designed,  developed,  and  tested  to  meet  the  specific  training 
needs  of  the  ARNG  with  direct  applicability  to  the  AC.  For  the 
RC  it  may  be  used  at  home,  in  the  armory  or  state  academies  for 
individual  study  via  modem,  or  in  a  networked  configuration.  It 
can  also  be  used  as  a  supplement  to  existing  courses  and  in  local 
learning  centers . 

AC  personnel  can  use  it  at  home  for  individual  study  via 
modem,  as  a  part  of  or  following  the  Basic  and/or  Advance  Courses 
prior  to  assignment  as  a  battle  staff  officer.  It  can  be  used  in 
operational  units  or  in  unit  education  centers  in  individual 
study  or  networked  configuration.  It  can  also  be  used  as  an 
addition  to  existing  professional  education  programs. 

This  project  started  on  individual  battalion  training 
modules  for  Reserve  Component  personnel .  It  can  be  expanded  to 
the  AC,  and  upward  to  brigade,  to  include  in  its  content  the 
greater  synchronization  responsibilities  of  personnel  in  the 
brigade  battle  staff.  The  applicable  successes  and  lessons 
learned  from  the  BN-BSTS  as  well  as  the  parent  program,  SIMITAR, 
are  available  to  enhance  the  individual  training  and  performance 
of  all  battle  staffs. 
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Appendix  A 

HARDWARE  AND  SOFTWARE  CONFIGURATION 

Hardware  and  software  (installed  and  configured)  were 
provided  for  eleven  student  stations  and  the  training  management 
system.  Specifications  were  finalized  to  permit  procurement  of 
additional  systems  at  a  later  date.  Requirements  include 
warranty,  technical  support,  and  on  site  maintenance  support. 

Student  Station 

Hardware ; 

486  DX2  Computer;  66  Mhz  processor 

8-16  Mb  RAM,  expandable  to  32+Mb;  70ns 

210-500  Mb  Hard  Disk 

16  bit  local  bus 

101  Key  Enhanced  Keyboard 

Desk  top  cabinet 

Serial  Mouse  with  driver 

Action  Media  II  (DVI)  card  with  driver/or  compatible  (must 
support  .AVS  files/compression  format) 

Sound  Blaster  16  Sound  board  with  driver/or  compatible,  and 
speakers 

150K/300K  double  spin  CD-ROM  or  equivalent  3-1/2",  1.44  Mb  disk 
drive 

9600bps  modem  with  lOBT  RJ45  connector 

Spider  Graphics  card,  or  equivalent,  with  driver  to  handle 
640x480,  256  color  and  14"  monitor 

Software :  DOS  6.21;  Windows  for  Workgroups  3.11;  Icon  Author; 
Training  Management  System  (TMS) 

Accessories :  Surge  Strip;  10BT/RJ45  modular  plug  with  10'  cable; 
Mouse  Pad 

Manager  PC 

33Mhz,  or  faster,  486  PC,  16Mb  RAM 
200Mb+60Mb  per  subject,  disk  storage 
Ethernet  adapter 
2  Shiva  net  modems 

Author  Station 

66Mhz,  or  faster,  486  PC,  16Mb  RAM 
Action  Media  II  (w/capture) 

Sound  Blaster  16  (w/speakers) 

500  Mb  disk  storage 
Ethernet  Adapter 

150K/300K  double  spin  CD-ROM  or  equivalent 

Spider  Graphics  card,  or  equivalent,  with  driver  to  handle 
640x480,  256  color  and  14"  monitor 
2Gb  Tape  Drive 
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Appendix  B 


INTRODUCTION  TO  BDM'S  ENVIRONMENT  FOR  MULTIMEDIA 
INTERACTIVE  INSTRUCTION 


The  TMS  Concept 

The  Training  Management  System  (TMS)  as  an  Environment  for 
Multi-Media  Interactive  Instruction  (EMMii)  has  been  developed  by 
BDM  as  an  enabling  environment  for  multi-media  computer  based 
training  (CBT)  development  and  delivery.  The  goals  of  TMS  are: 

To  provide  a  state-of-the-art  environment  where  trainers 
can  develop  computer-based  lessons  for  students,  incor¬ 
porating  high  quality  stills,  full  motion  video,  and 
audio  in  the  lessons. 

To  assist  the  Training  Staff  in  the  management  of  their 
curriculum  and  materials. 

To  automate  the  administration  of  training. 

To  allow  students  to  proceed  at  their  own  pace  and  to 
refresh  themselves  as  necessary. 

To  provide  on-line  testing  in  the  subjects. 

To  record  student/lesson  performance  data  in  an 
accessible  form,  and 

To  acquire  data  about  the  lessons  themselves. 

TMS  uses  commercial  off-the-shelf  (COTS)  software  (S/W) 
integrated  into  a  system  that  allows  trainers  and  administrators 
to  make  good  use  of  the  specific  functionality  of  the  packages. 
The  Microsoft  Windows  (TM)  environment  then  provides  a  familiar 
look  and  feel  across  the  packages . 


TMS  Hardware  and  Software 

The  original  TMS  was  developed  for  a  client  on  a  486  PC 
platform  under  OS/2 (TM) .  However,  BDM  has  ported  this  system  to 
a  Windows  (TM)  environment.  The  hardware  and  software  discussed 
here  are  for  the  Windows  platform. 
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The  hardware  architecture  for  this  system  is  illustrated  in 
figure  B-1.  TMS  computers  are  all  486  PCs,  attached  through  a 
local  area  network  (LAN)  and/or  a  wide  area  network  (WAN)  using 
the  computer's  internal  or  external  modem,  and  communicating  via 
Windows  for  Workgroups  (TM)  (WFW) .  The  lessons  are  stored  on 
magnetic  or  optical  media  serviced  through  the  TMS  "Manager" 
computer.  The  Author,  Instructor,  and  Student  Stations,  are  486 
PCs  also  running  WFW.  The  Author  and  Student  Stations  should 
preferably  be  equipped  with  multi -media  hardware  and  software  for 
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full  motion  video,  high  quality  stills,  and  audio^.  We  recommend 
the  Icon  Author  (TM)  application  software  package  for  lesson 
development  and  display. 

Figure  B-2  provides  brief  descriptions  of  TMS  functional 
areas.  More  detailed  descriptions  are  provided  later  in  this 
document . 


Functional  Area 

Description 

Training 

Information 

System  (TIS) 

This  is  the  database  interface  on  the  TMS  Manager.  It  Maintains 
data  on  lessons,  subjects,  courses,  student  histories,  schedules, 
etal.  for  administrators,  instructors,  and  authors. 

Author  Station 

This  is  a  collection  of  software  centered  around  the  authoring 
package  that  gives  access  to  files  stored  on  the  Manager  and 
allows  an  Author  to  create  and  maintain  lessons. 

Instructor  Station 

The  Instructor’s  functions  are  performed  through  the  TIS 
which  can  be  accessed  remotely  through  another  PC. 

Student  Station 

This  software  allows  the  student  to  log  in  and  then  manages 
the  student’s  access  to  lessons.  This  software  also  ensures 
that  results  files  are  sent  to  the  database  (TIS). 

Storage  Manager 

This  software  resides  on  the  Manager  PC.  It  organizes  and 
tracks  the  materials  that  make  up  the  various  CBT  lessons. 

It  can  provide  an  entire  lesson,  or  allow  authors  to  browse 
for  individual  files. 

Communications 

BDM’s  software  uses  Windows  for  Workgroups  (TM)  to 
communicate  between  stations. 

Figure  B-2,  TMS  functional  areas 

Digital  Video  Interactive  (TM)  (DVI)  is  the  most  robust  way  to  do  this.  However, 
other  digital  audio/video  options  such  as  Soundblaster  (TM)  cards  and  Video  for  Windows  (TM)  can 
also  be  used. 
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All  of  the  functional  areas  are  interrelated  and  communicate 
with  each  other  via  messages,  files,  and  database  queries. 

Figure  B-3  illustrates  the  interactions  between  the  Authoring 
Station,  the  Storage  Manager,  the  TIS,  the  Instructor,  and  the 
Student  Station. 
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Figure  B-3,  Interactions  of  Functional  Areas 
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An  author  can  get  any  previously  created  files  and  templates  from 
the  Manager  while  on  his  PC.  Lessons  are  created  using  the 
multi-media  authoring  package  IconAuthor (TM) .  An  author  can 
actually  work  on  their  lesson  locally  without  manager  interaction 
once  all  the  necessary  files  are  downloaded  to  the  authoring 
station.  When  the  lesson  is  completed,  procedures  on  the 
authoring  station  are  used  to  check  the  lesson  back  onto  the 
manager  where  the  lesson  and  its  associated  files  are  stored. 

The  TIS,  which  has  knowledge  of  the  lessons  and  their 
suggested  order  within  the  course,  allows  an  instructor  to 
schedule  students  for  particular  lessons  and  at  specific  student 
stations.  The  storage  manager  and  the  student  station  then 
manage  the  download  of  the  lesson  files  and  the  students'  own 
data  (bookmark)  to  the  student  stations. 

When  a  student  takes  a  lesson,  the  results  and  bookmark  are 
collected  at  the  prescribed  student  station.  Upon  completion  of 
a  student's  session,  the  results  are  transmitted  to  the  database. 
If  no  other  students  are  scheduled  for  the  same  station/lesson 
within  24  hours,  the  lesson  is  automatically  deleted  from  the 
student  station  upon  completion.  (Lessons  can  be  made  permanent 
on  a  student  station  as  well.) 

TMS  Course  Structure 


The  highest  level  of  organization  within  TMS  is  a  course.  A 
course  consists  of  subjects,  and  the  subjects  are  made  up  of  one 
or  more  lessons.  The  organization  is  illustrated  in  figure  B-4. 


Normally  only  one  author  will  work  on  a  lesson.  The  lesson 
consists  of  a  multimedia  presentation  produced  by  an  author  as 
well  as  text  files,  audio  files,  stills,  motion  video,  and 
graphics  files  used  in  the  presentation.  All  of  these  files,  as 
shown  in  figure  B-5,  become  associated  with  the  lesson  and  are 
stored  on  the  manager  PC  after  the  lesson  is  checked  in. 

The  files  are  downloaded  to 
users  (authors,  instructors, 
and  students)  whenever  the 
lesson  is  referenced.  The 
display  type  and  suffix  of 
the  files  are  maintained  by 
the  storage  manager  to  allow 
identification  at  the  other 
levels  when  downloaded. 


Maintenance  of  files 
and  versions  of  files  are 
performed  by  the  storage 
manager  (on  the  manager  PC) . 
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SCRIPTS  (ICONWARE  AND  PATHS) 

AUDIO  (DVI  AND  WAVE  FILES) 

~y  VIDEO  (DVI  FILES) 

€-7  GRAPHICS  (BMP  AND  PCX) 

*17  /  TEXT  (USER  FILES  AND  lA  “SMART  TEXT”) 


ANIMATION  (SCRIPTS) 

VARIABLES  (TEXT  FILES  FOR  USE  BY  SCRIPTS) 


Figure  B-5,  A  lesson  and  associated  files 


Common  files  may  be  shared  by  lessons  within  a  subject,  by 
subjects  within  a  course,  or  by  courses  within  the  entire  system. 

TMS  Functionality 

The  following  provide  somewhat  more  detailed  descriptions 
of  the  TMS  functions  than  were  provided  in  figure  B-1. 

Author  Station  Software _ 

Most  of  TMS  is  interactive.  Figure  B-6  shows  a  typical  TMS 
screen.  This  particular  screen  is  used  by  authors  to  enter  short 
descriptions  of  their  files,  in  order  to  make  browsing  more 
meaningful  when  the  lesson  is  stored. 


_ Project  File  Description 

Description:  SO  Characters  Maximum _  . 

I  Questions/Ansrs  on  E-Mail  1 


Common  to 

BACKGRND.  BMP 
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OAH 

BOOKMARK.IW 

1 

OCourse 

BOOKMARK.VAR 

■ 

OSubject 

CHAPTERS.VAR 

m 

•Lesson 

CHKBKMK.IW 

1  i 
iu 

Q  Reference  Titlel  Sort  by  Q  File  Name  #  File  Type  H 

I  d:\WFWVprojdsc.tnis  ;  |  |  Select  Project  | 


I  Display  Required  Fliesl  |  Accept  |  1  Cancel  |  |  Help  |  (  6k  | 


Figure  B-6,  A  typical  TMS  screen 
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In  addition  to  the  scripting  functions  provided  by  the 
IconAuthor  (TM)  package,  the  TMS  author  station  software  provides 
the  following  specific  functions: 

Temporary  storage  and  retrieval  of  lessons 

Fetching  files  from  previously  stored  lessons 

Checking  common  file  usage 

Entering  descriptions  of  lesson  files 

Checking  lessons  in  and  out 

Creation  of  a  new  lesson 

An  author  may  also  want  to  use  the  TIS  to  look  at  course 
organization  as  well  as  student  results.  The  TIS  can  be  accessed 
through  the  author  administration  functions. 

Along  with  the  TMS  author  S/W,  special  IconAuthor  templates 
and  subroutines  have  been  developed  to  allow  authors  to  retrieve 
and  use  data  provided  by  TMS  (such  as  the  bookmark,  course/ 
subject /lesson  names,  et .  al . )  and  to  provide  data  back  to  TMS 
upon  completion  (scores,  answers,  updated  bookmark,  student 
assessments,  et .  al . ) .  Other  special  templates  have  been 
developed  to  "automate"  some  standard  procedures  (use  of  the 
bookmark,  certain  test-types,  et.al.)  In  addition  BDM,  in 
concert  with  subject  matter  experts  and  training  developers  can 
develop  custom  templates  and  shells  to  be  used  throughout  the 
organization's  courseware,  to  give  it  a  standard  look  and  feel. 


Instructor  Station  Software 

Instructors  perform  their  work  through  the  TIS .  The  TIS  can 
be  accessed  through  both  the  author  administration  and  the 
instructor  PCs.  The  TIS  allows  an  Instructor  to  schedule  student 
sessions,  view  results,  look  at  course  structure,  et .  al . 

The  other  TIS  functions  used  by  an  instructor  include: 

Querying  the  status  of  a  student  station 

Adding  and  deleting  permanent  lessons  from  student 
stations 

Aborting  a  student's  session 
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The  student  station  software  consists  of  one  log-in  screen 
which  is  always  displayed  when  the  station  is  not  in  use.  In 
addition,  it  contains  several  background  processes  that: 

Receive  "schedule"  messages  from  the  database  and 
initiate  lesson  downloads 

Ensure  that  students  receive  the  lessons  for  which  they 
are  scheduled 

Retrieve  student  test  results  and  bookmarks  and  log  them 
to  the  "Host"  computer  (either  the  Manager  Computer  or 
the  Instructor  Station) 

Cleanup  the  student ' s  results  and  course  materials  after 
completion . 

The  Training  Information  System _ (TIS).  . 

The  TIS  serves  several  purposes  within  TMS .  Its  primary 
function  is  to  provide  course  administrators  and  instructors  with 
information  about  courses  that  are  available  and  information 
about  employees  training  activities.  The  TIS  also  maintains 
detailed  records  on  student  training  performance  and  on  student 
assessments  of  the  training  received.  In  addition,  the  TIS 
assists  the  instructor's  scheduling  of  students  for  particular 
lessons  at  specific  stations  for  specific  times;  thereby, 
avoiding  conflicts  with  student  time  and  station  scheduling, 
maximizing  utilization  of  stations,  and  helping  to  assure  that 
students  receive  training  in  the  proper  sequence.  Figure  B-7 
shows  a  typical  TMS  TIS  screen,  this  one  reporting  on  the 
students  who  have  taken  a  particular  lesson. 


TMS  uses  the  Microsoft  Access  (TM)  relational  database 
management  system  software.  Using  Access  provides  a  similar  look 
and  feel  to  the  other  TMS  functions.  The  TIS  consists  of  over  50 
screens  and  reports  that  allow  a  user  to: 

Add/delete  students 

Organize  curricula 

Schedule  student  training  sessions 
View  training  status  by  student  or  by  lesson 
View  exam  results  by  student  or  by  lesson 
Archive  course  materials 

The  Storage  Manager 


The  purpose  of  the  TMS  storage  management  module  is  to 
provide  a  transparent  disk  storage  system  where  users  on  any 
station  are  unaware  of  the  actual  storage  structure  use  for  TMS 
lesson  files.  The  storage  manager  provides 

an  efficient  storage  and  retrieval  system  for 
individual  TMS  lesson  files 

an  efficient  system  for  transferring  complete 
lessons  between  PC  workstations  and  the  manager  system 

Given  the  probability  that  the  lessons  developed  will 
undergo  changes  with  time,  configuration  management  of  the 
courseware  is  performed  by  the  storage  manager.  The  software 
also  allows  designation  of  files  as  common  to  several  lessons, 
decreasing  storage  space,  and  making  global  changes  easier  to 
perform. 

Communications 

The  TMS  communications  software  resides  on  each  PC  in  the 
system.  It  uses  Windows  for  Workgroups  to  perform  message  and 
file  transmission  over  an  existing  LAN  or  WAN  through  the  use  of 
a  modem  and  standard  telephone  lines.  In  addition,  small  local 
databases  are  used  by  TMS  S/W  to  allow  machines  in  the  system  to 
access  data  about  what  is  happening  on  another  PC  (such  as 
instructor/student  station  interactions). 
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APPENDIX  C 


STANDARDS  USED  FOR  THE  DEVELOPMENT  OF  ARNG 
COMPUTER  BASED  INSTRUCTION 


Tpl-r-nHuction 

Earlv  in  this  program,  BDM  in  collaboration  with  PRC  developed 

a  r.af  TnsVru" 

TcEirtS  be  uteTi^’blth^the  Battalion  Battle  Staff  Training  System 

BN  LtS)  and  the  Forward  Support  Battalion  (Combat  Service  Support) 
(BN-BSTS;  ana  one  ru  ^  National  Guard.  It  was 

expiSJl  tlTat  in  bftt  tfese  la°s"es"cBI  woid  be  used  in  conjunction 
with  other  instructional  media  (hands-on,  paper  base  , 

Since  the  initial  development,  both  teams  have  continued  to 
refine" thei^  standards,  though  not 

standards  used  by  BDM  in  and  sent  to 

?):fS™orrSa“L\?aiSnrMaSagementSy^ 
based  on  EDM's  commercial  product  bPmii 
Multi-Media  Interactive  Instruction. 

The  areas  addressed  for  CBI  Delivery  Standards  are  ^^^tandTr? 

C-1  Descriptions  and  details  of  the  components  of  each  standard 

Le'conJSSed  below.  More  detailed  information  is  included  in  the 
enclosures . 

TriRf.  ruction 

nbj actives. 

The  enabling  knowledge  for  entrance  into  ^ V^JJs^Basic 

basic  literacy  with  computers  and  completion  °fthe  Officers  Ba 
Course  (OBC)  for  the  respective  branch  officer  for  ^11 
except  S3  and  XO  where  completion  of  the  advanced 
reauired  Additionally,  officers  are  required  to  be  brancn 
qS2nf?ed  for  their  gride  (eg.  served  as  a  platoon  leader  for  SI, 
S4 . company  commander  for  S3 . )• 

Prior  to  beginning  a  specific  staff  officer 
course,  all  members  of  the  battle  staff  will  complete  the 
in  the  Common  Core  Course. 
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INSTRUCTION 

Objectives 

Role  of  Scenario  and  METT-T  Factors 
Course  Structure 
Course  Mapping 
Strategies 

Types  of  Interaction 
Role  of  Mentor 
Motivation 

Principles  for  Media  Use 
ASSESSMENT 

Diagnostics  and  Prescriptions 
Testing 

Feedback  on  Test  Performance 
Remediation 

LOOK  AND  FEEL 

Colors 

Fonts 

Button  Placement 
Borders 

Screen  Layout (s) 

FUNCTIONS 

Navigation 

Glossaries 

Accessories 

MANAGEMENT 

Data  to  be  collected 

Data  format 

Control  over  progression 
Support  system 
Reports 

Guides  (for  student,  instructor..) 

Delivery  mechanisms 

Security  


Figure  C-1  CBI  Delivery  Standards 


Ths  thirs©  Isvsls  of  Iss-irninQ  foir  ©  Issson  inci'uiis  fundatnsntal 
(individual  definitions  and  components  of  the  siaff 
functional  area) ,  individual  problem  solving  (give  a  problem, 
apply  knowledge,  produce  a  behavior),  and,  initially,  collective 
training  (issue  essential  portions  of  a  brigade  operations  order 
or  warning  order,  apply  basic  skills,  produce  individual  staff 
output, and  the  battalion  warning  order  or  operations  order) . 

(Based  on  guidance  received  in  the  August  1993  IPR,  collective 
tasks  were  deleted  and  only  those  individual  tasks  required  for  a 
staff  officer  to  enter  collective  training  were  retained) . 

Collective  tasks  are  to  be  initiated  in  a  future  program  to  tram 
dyads  and  triads  (small  staff  teams)  and  the  SIMUTA  program. 

The  form  of  training  objectives  is  Task  -  Condition 
Standard. 

The  organizational  competency  model  follows  Olmstead's  model, 
while  we  "greened  up"  the  explanation  to  ensure  it  is  received  to 
enhance  the  instruction. 

The  instruction  allows  students  to  take  the  subject  "for 
credit",  in  which  case  test  results  are  recorded.  However,  the 
student  may  also  choose  to  take  a  subject  for  reference,  merely 
peruse  a  subject,  in  which  case  some  tests  will  not  be  available 

to  him. 

Role  of  Scenario  and  METT-T  Factor.^ 

Common  scenarios,  based  on  the  NTC  and  synchronized  with  the 
SIMUTA  and  CSS  programs,  drive  the  basic  skills  and  problem  solving 
stages  of  instruction.  The  common  scenario  normally  drives  stages  I 
and  II;  or  scenarios,  tailored  for  the  individual  training 
objective,  are  adapted  for  stages  I  and  II,  when  required. 

The  factors  of  enemy,  troops,  and  terrain  [and  weather]  are 
stabilized  for  training  in  all  stages.  However,  the  factors  of 
mission  is  varied  for  offense  and  defense  while  time  is  the  key 
factor  that  is  varied  to  create  the  ability  to  perform  under 
increasing  factors  of  stress  and  decreasing  factors  of  time. 

Course  Structure  and  Mapping 

The  highest  level  of  structure  is  the  course  (SI,  S2,  S3,  ... 

SIGO) .  Courses  are  10-50  hours  duration.  The  next  (lower)  level  of 
structure  is  the  subject.  A  subject  is  a  grouping  of  lessons.  (See 
figure  C-2) .  The  subject  is  a  piece  of  IconAuthor  courseware  and 
associated  multimedia  files.  Most  subjects  are  of  1-2  hours 
duration  (in  the  CBI) .  Within  the  lesson  there  are  topics.  The 
common  core  course  map  shows  a  specific  example  of  such  a  structure. 
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Courseware  and  Associated  Multi-media 


LESSON  3A 

LESSON  3B  1 

1 

1 

1 

Figure  C-2  General  TMS  curriculum  structure 

Each  course  is  a  mixture  of  text -based  material  and  CBI .  All 
assessment,  performance  measurement  and  diagnostic  testing  is  done 
within  the  CBI.  There  is  a  short  (5-10  minute)  introductory  CBI 
subject  for  each  course  and  lesson  as  well  as  a  pretest  (more  on 
testing  later) .  The  lessons  themselves  are  available  through  a 
menu  in  the  CBI.  The  introduction  stresses  the  importance  of  the 
material  to  the  officer's  job  and  how  the  particular  material  fits 
into  the  overall  curriculum.  It  also  indicates  what  reading  should 
be  done  prior  to  the  taking  the  exam.  For  some  lessons,  the 
introduction  and  exam  are  the  only  components  of  the  lesson  within 
the  CBI.  Other  lessons  include  additional  CBI  material  (tutorials, 
practical  exercises,  or  examples  from  lessons  learned)  to 
strengthen  the  learning.  Such  lessons  are  indicated  by  a  check 
mark  on  the  course  map.  There  is  more  on  testing  philosophy, 
navigation,  and  structure  in  later  sections. 

Strategies 

The  specific  strategy  applied  to  achieve  a  specific  training 
objective  was  selected  by  the  development  team  SMEs  in 
conjunction  with  the  instructional  designer  following  the  model  of 
design  and  development  upon  which  each  team  member  was  trained. 

The  strategies  applied  in  the  course  development  process  are: 
tutorial,  drill  and  practice,  guided  discovery,  problem  solving, 
simulation,  games,  and  cooperative  learning.  Practice  is 
incorporated  to  the  maximum  extent  possible  to  attain  the  training 
objective . 

Training  for  the  development  team  included  the  best 
application  of  a  strategy  to  a  specific  training  objective.  The 
SME  development  team  was  also  trained  in  behaviorally-oriented 
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principles  of  instruction  which  are  appropriate  to  cfonsider  in 
designing  specific  instructional  strategies. 

Types  of  Interaction 

For  each  strategy  selected,  the  SME  has  designed  the 
appropriate  type  of  interaction  to  best  achieve  the  training 
objective.  Types  of  interaction  include  recognition,  recall, 
comprehension,  application/analysis,  synthesis,  and  evaluation. 

The  section  on  feedback,  diagnostics,  and  testing  has  more  on 
this . 

Role  of  the  Mentor 

Clearly,  the  optimum  method  to  achieve  a  high  performing 
staff  member  and  battle  staff  [particularly  in  high  order 
cognitive  tasks]  is  through  the  availability  of  a  mentor  during 
the  process.  As  to  whether  this  program  can  now,  or  in  the  future, 
acguire/af ford  a  mentor  has  been  discussed  and  explored  during  the 
period  of  the  contract.  Regardless,  an  instructor  or  trainer  is 
strongly  recommended  and  can  be  made  available  through  the  E-Mail 
and  Chat  functions  provided  through  the  TMS  and  Windows  for 
Workgroups  (WFW) .  (See  section  on  support  systems,  below.) 
Students,  at  any  time,  have  the  ability  to  pose  a  question  that 
will  ultimately  be  posted  for  response  by  a  mentor,  instructor,  or 
trainer.  As  the  battle  staff  progresses  through  the  course 
structure  (stages  I  -  III)  ,  there  may  be  an  increased  need  for  the 
mentor,  instructor,  or  trainer. 

Motivation 

Techniques  applied  to  motivate  the  student  (and  overcome  the 
commonly  observed  attrition  effect  in  distance  learning)  by  the 
SME/design  team  include  humor,  fantasy,  games,  challenges,  and 
competition  with  self  and/or  others.  These  techniques  are  applied 
based  on  the  learning  stage,  type  of  skill  and  instructional 
method. 

Principles  for  Media  Use 

Use  of  media  is  tailored  to  the  training  objective  and  to 
meet  the  test  of  effectiveness  (training  and  cost) . 


C-5 


video  is  used  judiciously  for  key  points,  to  gain -attention, 
or  for  motivation.  It  is  applied  for  demonstration  of  principles, 
equipment,  lessons  learned,  to  provide  a  historical  perspective, 
and,  to  a  minor  extent,  for  variety. 

Most  screens  use  audio  to  emphasize  key  points  and  supply 
additional  information  to  the  learning.  A  male  voice  is  used  for 
the  target  population.  Existing  (TASC)  audios  or  videos  have  been 
used  in  some  cases;  however,  most  have  been  created  by  the  SME 
specifically  to  support  the  learning  objective  . 

Text  is  used  on  most  every  screen  and  amplified  with  audio, 
video,  graphics,  photo  stills,  or  animation. 

Graphics  and  photo  stills  are  used  to  anchor  a  learning 
objective  and  apply  doctrinal  publications  (Field  Manuals,  ARTEPS, 
and  Tactics,  Techniques  and  Procedures)  to  ensure  the  officer  can 
quickly  refer  to  the  doctrinal  publication  in  field  application. 

Animation  is  used  judiciously  for  motivation  or  variety. 

Level  of  reading  is  a  10th  grade  level  of  reading. 

ASSESSMENT 

Diagnostics  and  Prescription 

Each  subject  begins  with  a  diagnostic  test  (pretest)  that  is 
identical  to  the  final  subject  examination.  The  student  is  not 
given  the  correct  answers  to  the  pretest  questions.  However,  he 
is  shown  which  questions  he  missed  and  is  told  (on  the  main  menu) 
the  lessons  he  did  well  enough  on  to  skip  over  during  the  conduct 
of  the  subject.  If  the  student  desires,  he  has  the  opportunity  to 
complete  lessons  on  which  he  has  demonstrated  mastery  through  the 
diagnostic  pretest. 

Testing 

There  are  three  types  of  tests:  quizzes,  lesson  exams,  and 
pretests/final  exams.  Quizzes  are  graded  to  provide  the  student  a 
measure  of  his  performance;  but,  results  are  not  stored  in  the 
database  for  future  analysis.  Exams  (lesson  and  final)  and 
pretests  are  graded,  issued  a  percentage  score,  identified  as  to 
pass/fail,  and  stored  for  analysis  and  review.  A  student  may  take 
any  quiz,  lesson  exam,  or  final  as  many  times  as  he  wishes.  Each 
pretest  may  be  taken  only  once . 

Each  subject  begins  with  a  pretest  that  the  student  must  take 
before  taking  the  subject  for  credit.  The  student  will  take  this 
test  only  once .  The  TMS  tracks  that  the  pretest  has  been  taken  and 
does  not  permit  it  to  be  re -taken.  If  the  student  passes  the 
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pretest  (achieves  a  score  of  80%  or  better  on  each  I'esson  in  the 
subject),  he  is  considered  to  have  passed  the  subject,,  and  can 
either  proceed  to  another,  or  continue  with  the  one  at  hand. 
Students  may  always  return  to  a  subject  they  have  taken  previously 
or  review  or  additional  study. 

There  is  also  a  final  exam  (mastery  exam)  for  each  subject. 

The  final  is  identical  to  the  pretest  (except  that  the  questions 
are  re-ordered  and  feedback  is  given  as  to  both  the  correct  answer 
and  where  the  student  can  gain  the  information  to  learn  the 
objective  measured  by  the  question)  and  covers  the  entire  subject. 
A  minimum  score  of  80%  is  required  for  mastery  before  the  student 
is  considered  to  have  "passed/receive  a  GO"  on  the  subject. 

Each  lesson  has  an  associated  lesson  examination.  This  exam  _ 
is  graded  and  recorded  in  the  data  base.  A  minimum  score  of  80%  is 
required  for  mastery.  Mastery  of  the  material  is  required  before 
the  CBI  will  advise  the  student  to  move  to  the  next  lesson. 

The  examination  format  is  multiple  choice  or  a  derivative 
thereof  in  most  cases,  as  the  computer  cannot  [easily  or^ 
accurately]  grade  subjective  type  examinations.  Derivatives  of 
the  multiple  choice  include  matching,  sorting,  choose  from  a  list, 
choose  spots  or  locations  on  a  map,  or  graphic,  or  use  of 
fill -in- the -blank. 

Most  lessons  also  contain  quizzes  or  practical  exercises 
(PEs) .  The  PEs  may  or  may  not  have  "correct"  answers  associated 
with  them.  Most  have  only  "suggested"  or  "A-Way"  answers  designed 
to  encourage  the  learning  of  the  cognitive  concept  [or  model] 
behind  the  PE  for  application  under  different  factors  of  METT-T. 
Responses  to  questions  in  PEs  are  neither  graded  nor  recorded  to 
encourage  the  battle  staff  officer  to  take  risks,  try  new  concepts 
and  enhance  the  cognitive  learning  process. 

Feedback  on  Test  Performance  and  Remediation 

As  mentioned  in  the  section  above  on  testing,  there  are  three 
types  of  tests  used.  The  answers  and  scores  on  two  of  the  three 
types  are  recorded  within  the  TMS.  The  following  addresses  user 
feedback  on  each  type. 

Quizzes  and  exercises  are  designed  to  be  instructional  to  the 
user  in  terms  of  what  he  has  learned  or  still  needs  work  on. 
Immediate  [right  or  wrong  for  quizzes]  feedback  is  given  on  each 
question;  with  reinforcement  for  right  answers,  and  inferences 
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as  to  what  is  correct  for  wrong  answers.  In  some  cases  the 
question  is  re-asked  when  the  answer  is  wrong.  The  PEs  follow  a 
similar  philosophy. 

Fill-in-the-blank  exercises  are  used  somewhat.  The  computer 
grades  these  by  comparing  the  user's  response  to  a  list  of 
acceptable  responses. 

Exams  are  not  used  as  a  Go/No  Go  between  lessons  (based  on 
guidance  in  the  September  1993  IPR) ,  but  merely  as  a  suggestion 
for  student  progression.  Failure  on  a  lesson  exam  (not  meeting  the 
80%  threshold)  does  not  prevent  the  user  from  proceeding  to 
another  lesson.  On  the  lesson  exams,  the  user  is  provided  specific 
feedback  for  each  question  by  providing  the  correct  response  and 
additional  information  to  anchor  the  learning  objective. 

On  a  pretest,  the  user  is  shown  which  questions  he  missed  and 
told  which  lesson  (within  the  subject)  to  which  the  missed 
question  applies.  Based  on  the  areas  missed,  the  user  is  shown  the 
main  menu  with  thumbs  up/down  to  indicate  which  lessons  should  be 
re-examined  before  taking/retaking  the  final  exam.  (See  the 
section  below  on  navigation.) 

Similarly,  upon  completion  of  the  final  exam,  the  user  is 
shown  which  lessons  he  did  well  on.  He  is  also  given  individual 
feedback  (right/wrong  and  in  some  cases  an  explanation)  for  each 
question.  He  is  normally  shown  the  correct  answer  for  these 
questions,  although  in  some  cases,  the  specific  feedback  indicates 
the  correct  answer.  Additionally,  feedback  will  refer  the  student 
to  text/CBI  where  he  can  study  to  achieve  the  learning  objective. 

Look  and  Feel 

The  following  standards  reflect  what  is  readily  available 
within  the  Windows  and  the  IconAuthor  environment .  Templates  were 
created  to  make  much  of  the  navigation  and  functionality  more 
uniform  across  lessons,  subjects  and  courses  and  enhance  the  time 
required  to  produce  the  CBI. 
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Colors 


Menus  and  opening  screens  have  a  dark  gray  background  with 
silver  buttons.  Individual  screens  within  lessons,  _ introductions, 
0xercises  etc.  use  a  variety  of  colors  to  enhance  interest, 
readability,  and  maintain  student  attention. 

Text  is  in  a  variety  of  colors .  Menus  and  opening  screens  use 
black,  white,  silver,  navy,  and  maroon  (for  emphasis) .  Other 
screens  use  colors  that  provide  good  contrast  to  the  background . 
Use  of  florescent  colors  has,  normally,  been  avoided  because  these 
often  appear  to  pulsate  on  a  bright  screen  and  can  thus  be 
annoying . 

Fonts 

Arial  is  used  for  most  text;  12  point  and  14  point  are 
preferred  for  normal  text,  with  larger  sizes  for  titles.  In  some 
instances  10  point  arial  is  used  when  there  is  inadequate  space 
for  a  larger  font.  Buttons  are  in  a  standardized  font  across  the 
entire  system  (all  courses,  subjects,  and  lessons)  as  they  are 
based  on  templates  designed  for  the  entire  system. 

Button  Placement 

Navigational  buttons  are  at  the  bottom  of  the  screen  and  are 
consistent  throughout  all  the  courseware.  Size  and  placement  of 
buttons  are  discussed  in  the  section  below  on  screen  layout.  For 
PEs,  buttons  are  placed  in  the  logical  location  to  aid  student 
learning. 

Borders 

Screens  do  not  have  borders  on  them.  This  enabled  us  to  make 
full  use  of  the  space  available.  Many  screens  will  have  a  small 
text  field  or  icon  in  the  upper  left  corner  as  an  indicator  of  the 
subject  to  aid  the  student  in  navigation. 

Screen  Layout 

Menus  contain  no  more  than  eight  menu  buttons,  with  screen 
navigation  buttons  at  the  bottom.  They  are  generally  240  by  40 
pixels.  (Larger  if  needed) . 
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Menu  buttons  generally  correspond  to  lessons  or  topics. 
Normally  the  user  will  progress  through  the  lessons/topics  in  the 
sequence  designed  by  the  SME  and  as  shown  (top  to  bottom) .  Figure 
C-3  provides  a  sample  main  menu  from  the  IPB  subject  in  the  52 
course . 

To  aid  the  user  in  navigating,  the  main  menu  is  titled  with 
the  course  and 
subject,  as  in 
figure  C-3 . 


Submenus  are 
used  to  allow  the 
user  access  to 
individual  items 
within  a  lesson. 
These  include,  at  a 
minimum,  the  lesson 
introduction  and 
lesson  exam.  In 
some  instances, 
they  provide  access 
to  topic  material 
and/or  practical 
exercises . 


S2:  Intelligence  Preparation  of  the  Battlefield 


Lessons  you  have 
seen  before  or  done 
well  on  are  italicized. 


Thumbs  Up  indicate 
lessons  you  have 
already  passed. 


Previous  I  I  Rftptoy 


Figure  C-3  A  Sample  Main  Menu 


In  either  case,  the  same  look  and  feel  standards  mentioned 
above  apply.  The  submenu  contains  the  name  of  the  course  and 
lesson  instead  of  the  subject  and  lesson. 

Other  than  menus,  most  other  material  is  shown  on  a  screen 
such  as  figure  C-4.  Buttons  are  40x160  (Options  and  Replay)  and 
40x120  (Next  and  Previous) .  Not  all  buttons  are  available  all  the 
time.  Those  not  available  are  grayed. 
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The  Next  button  is  used  for  page  turning  when  need.ed.  Not 
all  screens  require  the  user  to  press  Next  to  get  to  the  following 
screen,  but  any  screen  with  a  substantial  amount  of  text  or 
diagrams  allows  the  user  to  choose  when  to  proceed. 

The  Previous  button 
is  the  reverse  of 

Next. 


The  result  of 
"Previous"  is  not 
necessarily  "the 
previous  screen" . 
For  example,  one 
may  show  a  diagram 
with  audio, 
followed 

immediately  by  a 
text  screen 
referring  to  the 
diagram . 


Determ'i1^roet@jla^fimprovie: 


OptiOM 


^;|Repiay^ 


Figure  C-4  A  Standard  Screen  with  Navigation  Buttons 

The  text  screen  contains  the  Next  button,  which  sends  the 
user  to  a  third  screen.  The  "Previous"  button  on  the  third  screen 
might  more  naturally  send  the  user  back  to  the  diagram  rather  than 
the  text . 

The  Replay  button  is  more  immediate.  It  is  used  to  replay  an 
audio  or  an  animation.  Again,  Replay  may  cause  the  user  to  replay 
several  screens . 

The  Options  button  is  a  catch-all.  It  produces  a  sub-menu  of 
options.  These  normally  include  a  Menu,  a  Glossary,  a  Pose  a 
Question  function,  and  a  Quit  function.  A  Cancel  button  removes 
the  options  from  the  screen.  Figure  C-5  provides  a  sample  of  what 
this  looks  like.  Again,  not  all  of  these  are  available  all  the 
time,  although  the  glossary  and  pose  a  question  functions  are 
available  most  of  the  time.  The  Menu  function  returns  the  user  to 
the  menu  on  which  the  current  item  (exam,  lesson,  topic,  exercise, 
etc.)  resides.  Other  standards  for  glossaries  etc.  are  discussed 
below. 

The  Glossary  and  Pose  a  Question  functions  are  not  available 
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idenliM'easdnMthatiu.nits  ;fail  .to, 
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l;a^ybiRti^Jo||l|i^p^^ts':-  ■  ;■ 
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:CanMt 


Iglpifaiy 


during  exams  or  the 
pre-test  or  final 
exams . 

In  the  case  of  a 
multiple  choice 
question,  50  by  50 
pixel  buttons, 
labeled  A-E  are 
used.  If  the 
question  has  fewer 
than  five  possible 
answers ,  unneeded 
buttons  are  not 
shown . 


Figure  C-5  Sample  of  the  Options  Menu 

For  most  questions,  the  Next  button  is  used  by  the  user  to 
register  his  response,  thus  allowing  him  to  change  his  answer 
before  it  is  recorded  and/or  checked. 

Functions 

Navigation 

Initially,  the  user  is  shown  the  screen  in  figure  C-6.  If  he 
has  never  taken  the  pretest  for  this  subject,  the  pretest  button 
is  available  (ungrayed)  and  the  final  exam  button  is  unavailable 
(grayed) .  If  he  has  taken  the  pretest  previously,  the  pretest 
button  is  grayed  and  the  final  exam  button  is  ungrayed.  In  either 
case,  the  user  may  look  at  the  introduction  or  go  to  the  main 
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menu.  If  he  chooses  the  main  menu,  he  is  asked  if  he''  wants  to  take 
the  subject  for  credit. 


Figure  C-6  The  Initial  Screen 

A  sample  main  menu  is  shown  in  figure  C-7.  This  one  shows  a 
user  who  has  not  passed  the  pretest,  but  has  either  passed  the 
lesson  exam  for  the  weather  analysis  lesson  or  passed  the  weather 
analysis  material  on  the  pretest.  (This  is  indicated  by  the 
"thumbs  up".)  In  figure  C-7,  the  third  lesson,  terrain  analysis, 
is  italicized,  indicating  the  user  has  seen  some  of  its  material 
at  a  previous  time.  This  user  has  not  yet  taken  the  final  exam. 
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Figure  C-7  The  main  menu  for  a  user  who 

performed  well  on  only  one  lesson  on 
the  pre-test 

Whenever  a  student  exits  the  CBI,  a  bookmark  is  placed  within 
his  local  database  (and  eventually  the  host  (Armory)  database)  so 
the  next  time  he  returns  to  this  subject,  his  main  menu  will  look 
the  way  it  did  when  he  last  exited  the  subject. 

The  lesson  menu  will  indicate  (via  a  thumbs  up)  whether  or 
not  the  lesson  has  been  passed  (either  via  the  lesson  exam,  via 
the  pretest,  or  via  the  final  exam.  If  the  lesson  has  not  yet 
been  passed,  the  item  on  the  menu  last  viewed  by  this  user  is 
italicized.  Figure  C-8  shows  the  lesson  menu  for  S3:  METT-T 
Analysis  where  the  user  has  not  passed  the  lesson  and  he  last 
viewed  the  Mission  Analysis  topic. 
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ntroduction  to  METT-T  Analy: 


Tactical  Decision-Making  Proo 


Once  the  user  has  taken  the  final  exam  (at  least  once) ,  the 
main  menu  will  indicate  (via  thumbs  up)  which  lessons  were  passed 
on  the  final. 


Mss/on  Ana/ysis 
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Each  course  has  a  glossary  designed  by  the  SMEs  to  aid 
learning.  The  glossary  is  available  through  the  Options  button. 
The  Glossary  itself  is  a  Windows  Help  File,  and  as  such  has  hot 
spots  and  keyword  searches  available.  The  first  page  of  such  a 
glossary  is  shown  in  figure  C-9.  The  user  may  return  to  the  last 
screen  he  was  on  by  using  the  Previous  button. 


ifCm  S3:  METT-T  Analysis 


Lessons  you  have 
seen  before  or  done 
weli  on  are  italicized. 

Thumbs  up  indicate 
lessons  you  passed 
on  the  Final  Exam. 


Figure  C-8  Lesson  menu  where  a  user  last  viewed  the  third  item 


Battle  Staff  Glossary 


File  Edit  Bookmark  Help 


Scion  to  fiiiaitKioHew^^^ 

Acranyms  aii^^MFaiiaik  iaaf  the  iS^  aitlifuttiin  aliove^^^ 

Iteim  ill  Blue  jMpiate  a  pBI  of^^  youuiay  fiMmpio  udfoi:™^ 

lisu^  the  glossiuyi — i-iLi_..j^;jiii:iT 


A.Ra^tions 

A/L  order 
abatis 

sc  c  eptab|lit^.  ^.aljfsis 
ac  c  omp.^yiiig  .suppHe  s 

ackxiov4&dg§ _ 


airfflyAiition  transfer  p  oint 
arnphibipiLJs.  op  eratipn 
analyze 
.^mex 

.^tip  er  s  pmel.  and/pr  .antimatenal  munitipnl 


antip er s omiel.irnp.toysd  c pnyentip.nal.rfflW 


acj  area  of  interest  (AI)  That  area  of  concern  to  the  commander,  including  the  area  of 
ac  areas  adjacent  thereto,  and  extending  into  enemy  territory  to  the  objectives  of  currer 
ad  operations.  This  area  also  includes  areas  occupied  by  enemy  forces  who  could  jeop 
ad  accomplishment  of  the  mission, 
ad  (CBI:  S2/1A2;  XO/3A2) 


Figure  C-9  Sample  from  the  glossary 

Other  accessories  include  a  Course  Map  and  the  Pose  a 
Question  function  on  the  Options  key.  These  are  discussed  below 
and  in  the  section  on  management. 

The  student  is  allowed  to  type  in  a  question  for  his 
trainer/mentor  via  a  Pose  a  Question  function  in  the  Options.  The 
question,  plus  some  information  about  where  in  the  subject  he  was 
when  he  asked  it,  are  written  to  a  file.  This  file  is 
automatically  sent  to  the  host  computer  (armory)  system  when  the 
student  uploads  results.  (See  support  systems,  below). 
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Management 

Dat-a  rollected  from  Training  Sessions. 

Every  exam  within  a  lesson  has  a  number .  This  number  along 
with  the  type  (lesson  exam,  pretest  and  final  exam)  is  recorded  in 
the  database,  as  are  all  the  user's  responses  to  computer-graded 
questions.  Thus,  the  TMS  has  access  to  what  answer  (A-E)  the 
battle  staff  officer  gave  to  question  number  3  on  Exam  2.  If  the 
battle  staff  officer  takes  an  exam  more  than  once,  each  iteration 
is  kept  as  a  separate  record. 

The  start /stop  date/times  for  each  student  session  and  the 
student ' s  bookmark  are  recorded  in  the  database .  Other  data  are 
also  maintained: 

Names  and  social  security  numbers  of  battle  staff  officers 

Current  progress  in  courseware . 

Courses  being  used  by  officers. 

Course  (s)  curriculum. 

Data  Format 

Attached  are  sample  screens  and  reports  from  the  TMS 
database.  These  should  give  a  reasonable  idea  of  the  type  of  data 
available.  Additional  data  are  available  in  the  TMS  user's  guide. 

The  TMS  does  allow  the  database  administrator  to  establish 
roles  for  users.  These  roles  then  determine  which  screens, 
functions  and  reports  that  a  user  may  access. 

Control  over  Progression 

Originally,  it  was  expected  that  the  TMS  would  control  the 
student's  progression  through  the  courseware  by  disallowing  access 
to  a  subject  if  its  prerequisites  had  not  been  passed.  Following 
instructions  we  received  during  the  September,  1993  IPR,  this  is 
no  longer  the  case.  A  student  can  access  any  subject  or  lesson 
for  which  he  is  scheduled  in  any  order.  The  student  will  be  told 
which  subject (s)  he  has  accessed  and/or  passed. 

What  distinguishes  a  course  from  a  subject  in  the  user's 
viewpoint  will  be  the  fact  that  within  a  subject  he  will  have 
menus  and  screens  that  allow  him  to  navigate  between  lessons  and 
topics.  Whereas  going  from  one  subject  to  another  will  generally 
involve  an  exit  from  the  specific  courseware  to  the  TMS  student 
environment . 

The  standards  discussed  in  this  document  (size,  content,  and 
structure)  are  for  a  Subject. 
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within  a  course,  the  student  progresses  from  subject  to 
subject.  Within  a  subject,  he  goes  from  lesson  to  lesson.  While 
the  terms  subject  and  lesson  are  consistent  within  the  student 
documentation,  within  the  Battle  Staff  TMS  (the  software  and  file 
Structure) ,  a  lesson  and  a  subject  are  the  same  thing. ^  This  is 
best  understood  by  thinking  of  each  (TMS)  subject  consisting  of 
one  (TMS)  lesson,  each  with  the  same  name.  The  TMS  thus  does  not 
know  about  the  lessons  within  the  subject's  iconware. 

The  user  is  registered  at  the  host  computer  (armory)  for  a 
course  (or  courses)  and  is  provided  with  a  schedule  disk;  CD 
ROM(s)  containing  the  courseware;  the  student  guide  containing  the 
text,  curriculum,  job  aids,  maps,  operations  order,  instructions 
for  assembling  the  computer,  etc.;  and  his  student  computer.  The 
student  then  can  take  his  computer  home  and  study  at  his  own  pace, 
in  the  sequence  of  his  choice,  and  become  a  proficient  battle 
staff  officer. 


Support  Systems 

Outside  the  courseware  is  the  TMS  student  environment.  The 
user  has  several  functions  available  here,  as  shown  in  figure  C- 
10 . 

The  user  is  able  to  connect  to  the  host  computer  via  a 
(Shiva)  net  modem  (located  at  the  armory  end)  and  appropriate 
modem  software  (S/W)  at  the  student's  machine.  The  connect  is 
performed  easily  via  the  TMS  Student  Station  S/W.  Once  connecte  , 
the  student  has  the  use  any  of  the  network  functions:  results 
upload.  E-mail,  and  chat,  via  the  TMS  Student  S/W.  Results  upload 
sends  data  from  the  student ' s  local  database  to  the  host  database 
at  the  armory  for  review  by  the  trainer.  Any  questions  he  may 
have  posed  are  also  sent  to  the  host  database.  The  E-mail 
facility  is  the  standard  one  provided  with  Windows  for  Workgroups 
(WFW) .  It  allows  the  user  to  send  mail  to  other  users  and  to 
receive  mail.  The  chat  facility  is  also  via  WFW.  It  allows  users 
on  the  network  to  converse  with  each  other  in  real  time  through 
text  messages. 
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The  user  is  also  able  to  find  out  where  in  the  bourse 
progression  he  stands  (lesson  summary)  and  what  he  should  take 
next.  This  is  available  through  the  TMS  student  environment,  not 
the  courseware . 


Subject 

Login 


File  View 

1 

iHl 

Help 

Show  Subject  Summary 

Copy  New  Student  Schedule 
Post  Results  to  Host 
Start  Mail 
Start  Chat  utility 
Unload  a  Subject 
Connect  to  Host  computer 
Disconnect  from  Host  computer 


Figure  C-10  Student  Functions  Available  in  TMS 
Reports 

Attached  are  some  standard  TMS  reports.  These  indicate  the 
type  of  data  available  to  the  trainer. 

Guides 

Each  course  contains  a  student  guide  that  includes  the  course 
text,  syllabus,  course  map,  list  of  lessons,  list  of  topics,  all 
training  objectives,  a  comprehensive  list  of  references, 
instructions  for  assembling  the  computer  and  instructions  in  case 
the  computer  doesn't  respond  as  expected.  Additionally,  the 
brigade  operations  order,  students  handouts,  job  aids  for  his 
smart  book,  maps  or  templates  required  to  complete  a  course  are 
included  in  the  guide.  The  student  guide  refers  the  student  to 
existing  doctrinal  publications  but  does  not  create  new,  or 
modified,  doctrinal  publications.  One  of  the  key  goals  of  the 
courses  of  instruction  is  to  enable  the  student  to  become 
proficient  in  both  the  knowledge  in  the  doctrinal  publications  and 
learn  to  quickly  refer  to  the  publications  in  a  field  environment. 
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There  is  an  instructor  guide  for  the  system  for'. use  by  the 
mentor,  instructor,  or  trainer.  The  instructor  guide  includes  all 
key  information  in  the  student  guides,  a  guide  to  how  to  conduct 
an  AAR,  and  the  references  for  each  topic  to  guide  the  student  in 
remediation.  Additionally,  instructions  on  how  to  operate  in  the 
TMS  environment  (scheduling  a  student,  monitoring  a  student's 
progress,  reports,  etc)  are  included  in  the  guide. 

Delivery  Mechanisms 

The  courseware  will  be  delivered  via  486  PCs  equipped  with 
digital  video  interactive  (DVI)  play  capability,  a  16 -bit  sound 
card,  and  speakers. 

Means  of  delivery  of  courseware  to  the  PC  is  via  CD. 

Security 

Security  at  the  student  level  was  deemed _ unnecessary . 

While  it  isn't  easy  for  a  student  to  modify  his  answers  before 
they  go  to  the  database,  it  is  not  impossible. 

As  mentioned  earlier,  the  TMS  allows  a  wide  range  of 
variation  in  access  to  data  among  users.  We  will  have  to  work 
with  the  Guard  to  establish  the  set  of  roles  used  and  who  will 
have  what  role. 


C-20 


Form  Edit  Records  Help 


Stud<bnts 


id:  1274804089 


Last  N  ame: :  jKasprqik 


First  Name:  '  I Jennifer 


Middle 


Position:';  lAdmin  CoorcTnatof 


D^artment:  |TS,  214 


Location:  |Ft  Benning 


Phbne:  1706-545-3734 


Figure  C-11  Student  data  screen 


Figure  C-12  Screen  for  adding/editing  course  information 
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Figure  C-13  Screen  for  adding/ editing  lesson  information 


Form  Edit  Becofds  ijelp 
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I  Common  Core 


Figure  C-14  Screen  for  scheduling  a  student  for  an  entire  course 
on  CD  ROM 
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Figure  C-15  Viewing  a  student's  schedule 
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Figure  C-16  Viewing  a  student's  status  in  a  lesson 
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Figure  C-18  Viewing  a  student's  exam  answers 


Figure  C-17  Viewing  the  history  of  a  given  lesson 
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01  •  1  beieve  1  had  sufficient  background  knowledge  to  prepare  me  for 

Ihislessoa 

02  *  The  posf-teri  was  an  accurate  assessment  of  rry  knowledge  on  this 
subject  1 

03 '  The  quizzes  and  exams  often  tested  irrelevant  or  unimportant 
information. 

04  •  The  C8i  portion  enhanced  inderstandhg  of  the  paper-based 
lesson  materi^  (riudent  guide,  FMs,  eta) 

05  •  The  computer  setup  and  administrative  procedues  were  loo  much 

uoine. 

r:s^ 


Figure  C-19  A  summary  of  student  opinions 
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Figure  C-20  Viewing  one  student's  opinions 
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R 

5.00 

5 

5 

1 

10 

After  completing  this  lesson,  I  believe  I  met  or 
exceeded  the  learning  objectives  as  stated  in  the 
student  guide. 

R 
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and  knowledge  required  to  perform  my  military  job. 

R 

5.00 

5 

5 

1 

12 

The  computer-based  instruction  was  challenging 
and  held  my  interest. 

R 

5.00 

5 

5 

1 

13 

This  lesson  was  well  organized  (key  concepts  were 
related  and  progressed  in  a  logical  sequence). 

R 

4.00 

4 

4 

1 

14 

There  was  too  much  material  to  try  and  learn  in  this 
lesson. 

R 

4.00 

4 

4 

I 

15 

There  was  too  much  repetition/redundancy  in  this 
lesson. 

R 

2.00 

2 

2 

1 
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Criterion 

Description 

Type 

Average 

Minimum 

Maximum  Standard 
Deviation 

Count 

16 

The  student  guide  was  easy  to  use  (instructions  were 
easy  to  understand). 

R 

5.00 

5 

5, 

1 

17 

This  lesson  should  have  more  examples  of  real  life 
application. 

R 

5.00 

5 

5 

1 

18 

The  vocabulary  and  educational  level  of  this  lesson 
were  too  elementary. 

R 

2.00 

2 

2 

1 

19 

The  computer-based  "exercises”  were  very  helpful 
in  learning  the  material. 

R 

5.00 

5 

5 

1 

20 

I  would  recommend  extensive  modification  to  this 
lesson  before  giving  it  to  other  students. 

R 

1.00 

1 

1 

1 

21 

I  believe  this  is  an  effective  teaching  method  for  the 
National  Guard. 

R 

5.00 

5 

5 

1 

22 

I  feel  more  confident  in  my  ability  to  perform 
combat  related  staff  functions  after  completing  this 
training. 

R 

5.00 

5 

5 

1 
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Opinion  Criterion  Summary 

18-Feb-95 

Criterion 

Description 

Type 

Average 

Minimum 

Maxifnum  Standard 
.  Deviation 

Count 

1 

I  believe  I  had  sufficient  background  knowledge  to 
prepare  me  for  this  lesson. 

R 

4.00 

4 

4 

1 

2 

The  post-test  was  an  accurate  assessment  of  my 
knowledge  on  this  subject. 

R 

5.00 

5 

5 

1 

3 

The  quizzes  and  exams  often  tested  irrelevant  or 
unimportant  information. 

R 

2.00 

2 

2 

1 

4 

The  CBI  portion  enhanced  my  understanding  of  the 
paper-based  lesson  material  (student  guide,  FMs, 
etc.) 

R 

5.00 

5 

5 

1 

5 

The  computer  setup  and  administrative  procedures 
were  too  much  trouble. 

R 

5.00 

5 

5 

1 

6 

In  the  future,  I  would  prefer  to  take  required 
instruction  using  CBI  instead  of  conventional 
correspondence  lessons. 

R 

5.00 

5 

5 

1 

7 

I  prefer  regular  classroom  instruction  to  this  method. 

R 

1.00 

1 

1 

1 

8 

The  paper-based  and  CBI  portion  of  this  lesson 
often  seemed  disjointed  and  unrelated  to  each  other. 

R 

2.00 

2 

2 

1 

9 

This  lesson  thoroughly  covered  the  training 
objectives  as  stated  in  the  student  guide. 

R 

5.00 

5 

5 

1 

10 

After  completing  this  lesson,  I  believe  I  met  or 
exceeded  the  learning  objectives  as  stated  in  the 
student  guide. 

R 

4.00 

4 

4 

1 

11 

I  now  have  a  much  better  understanding  of  the  skills 
and  knowledge  required  to  perform  my  military  Job. 

R 

5.00 

5 

5 

1 

12 

The  computer-based  instruction  was  challenging 
and  held  my  interest. 

R 

5.00 

5 

5 

1 

13 

This  lesson  was  well  organized  (key  concepts  were 
related  and  progressed  in  a  logical  sequence). 

R 

4.00 

4 

4 

1 

14 

There  was  too  much  material  to  try  and  learn  in  this 
lesson. 

R 

4.00 

4 

4 

1 

Criterion  Description  Type  Average  Minimum  Maximum  Standard  Count 


Deviation 

15 

There  was  too  much  repetition/redundancy  in  this 
lesson. 

R 

2.00 

2 

2 

1 

16 

The  student  guide  was  easy  to  use  (instructions  were 
easy  to  understand). 

R 

5.00 

5 

5 

1 

17 

This  lesson  should  have  more  examples  of  real  life 
application. 

R 

5.00 

5 

5 

1 

18 

The  vocabulary  and  educational  level  of  this  lesson 
were  too  elementary. 

R 

2,00 

2 

2 

1 

19 

The  computer-based  "exercises"  were  very  helpful 
in  learning  the  material. 

R 

5.00 

5 

5 

1 

20 

I  would  recommend  extensive  modification  to  this 
lesson  before  giving  it  to  other  students. 

R 

1.00 

1 

1 

1 

21 

I  believe  this  is  an  effective  teaching  method  for  the 
National  Guard, 

R 

5.00 

5 

5 

1 

22 

I  feel  more  confident  in  my  ability  to  perform 
combat  related  staff  functions  after  completing  this 
training. 

R 

5.00 

5 

5 

1 
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Appendix  D  '■ 

BETA  Test  Results 

BATTLE  STAFF  TRAINING  SYSTEM 
COMMON  CORE  BETA  TESTING  SUMMARY  AND 
ACTIONS  REQUIRED  3  November  1994 


I.  Demographic  Data;  Student  officers  are  ARNG  officers  serving 
as  Bradley  platoon  leaders  in  the  ARNG.  Officers  completed  lOBC 
one  week  prior  to  start  of  testing.  Average  age  was  24, 
education  level  was  college  graduate  with  one  graduate  student. 
Average  time  on  active  duty  (except  for  training)  was  0  with 
average  reserve  duty  time  3.25  years.  None  of  the  officers  had 
experience  at  the  NTC.  General  comments  from  all  officers  were 
negative  to  traditional  correspondence  courses  and,  to  a  lesser 
extent,  resident  training.  Comments  to  the  quality  of  Army 
publications  was,  overall,  negative. 


II.  Post  Course  Survey  Results: 

Of  the  15  questions  on  the  post  course  survey,  93%  were 
positive  (+)  on  the  course  of  instruction  with  7%  neutral  (0) ; 
there  were  no  negative  comments.  (For  positively  based  questions 
a  +  was  indicated  by  a  5  or  4 ;  for  negatively  phrased  questions  a 
+  was  indicated  by  marking  a  1  or  2;  3  was  the  neutral  range) . 

Overall  Comments: 

Course  was  a  good  supplement  to  the  FMs 

Very  good  and  challenging 

Suggestions : 

More  real  world  examples  and  (CBI)  exercises 

More  computer  instruction. 
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Subject  1  -  Intro  to  ALB 

TIME:  4.75  +  2  =  6.75/2=  3.37 

PRETEST:  60%  +  60%=120%/2=  60% 

FINAL:  50+83+83=216/3=72 

LESSON  EXAM:  70+83=153/2=76.5 
FIXES :  NONE 

COMMENTS:  EXCELLENT  QUIZ 

TOO  VERBOSE? 

ALL  INFORMATION  THERE  I  NEEDED 
Subject  2  -  DEFENSE  FUNDAMENTALS 
TIME:  4. 5+5=9. 5/2=  4.75 

PRETEST:  45% 

FINAL:  80+60+90=230/3=76.6 

LESSON  EXAM  2A:  80+72=152/2=76 

LESSON  EXAM  2B:  80  +  50=130/2=65% 

FIXES :  NONE 

COMMENTS:  I  UNDERSTOOD  THIS  SECTION 

VERY  LIMITED  CBI  -  GOOD  OUTLINE  OF  MATERIAL 
I  RELATE  BETTER  TO  THE  COMPUTER  THAN  THE  BOOK 
GOOD  EXPLANATION  OF  DEFENSE  FUNDAMENTALS . . . 
GOOD  QUIZZES  AND  TESTS 
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Subject  3  -  OFFENSE  FUNDAMENTALS: 

TIME:  1.5  +  3  =  4.5/2=2.25 

PRETEST:  60+53=113/2=56.5 

FINAL:  93+60+93=246/3=82 

LESSON  EXAM  3A:  95+36=131/2=65.5 

LESSON  EXAM  3B:  63+53=118/2=59 

FIXES:  -  3ATEXT  MENTIONS  QUIZ:  NOT  PRESENT 

3BTEXT  MENTIONS  QUIZ:  NOT  PRESENT 

COMMENTS:  COMPUTER  HELPED  PUT  THINGS  TOGETHER  FOR  ME 
LESSONS  ARE  GETTING  MORE  INTERESTING 
HISTORICAL  QUOTES  AND  REFERENCES  ... ADDED  VALUE 
GOOD  SEQUENTIAL  ORDER,  PROGRESS  UNDERSTANDABLE 


Subject  4A-  BOS  PART  I 

TIME:  6.75+  6  =  12.75/2=6.38 


PRETEST:  62+60=122/2=61 


FINAL:  53+87+70+90=300/4=75 


LESSON 

EXAM 

4A: 

48+96+60=204/3 

=  68 

LESSON 

EXAM 

4B: 

80+80=160/2= 

80 

LESSON 

EXAM 

4C: 

73+38=111/2= 

55.5 

LESSON 

EXAM 

4D: 

65+45=110/2= 

55 

FIXES : 

4A 

(Intell)  TOO  MUCH 

INFORMATION 

NEEDS  A  REVIEW 

NOTE:  THEY  ALREADY  TOOK 


S2 


COMMENTS:  NEED  FOR  MORE  CBI 

TESTS/QUIZZES  GIVE  ACCURATE  ASSESSMENT 
INFO  PRECISE,  NAVAL  GUNFIRE  MAY  BE  TOO  IN  DEPTH 
UNDERSTOOD  A  LOT  MORE  THIS  SECTION 
(ADA) NEW  CONCEPTS  I  AM  NOT  FAMILIAR  WITH 
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Subject  4B-  BOS  PART  II 

TIME:  6.5+7.45=  13.95/2=  6.9 

PRETEST;  62+87+68=  217/3=72.3 

FINAL:  53+87+80=220/3=73.3 

LESSON  EXAM  4E:  95+70=165/2=82.5 

LESSON  EXAM  4F1 :  60+75=135/2=67.5 
LESSON  EXAM  4F2 :  60+96=156/2=78 
LESSON  EXAM  4G:  84+75=159/2=79.5 

FIXES:  4G,  Too  long?  Break  into  2  pieces? 

The  hardest  section 

COMMENTS:  MORE  THAN  ADEQUATE  TO  COVER  MATERIAL 
(CSS)  MAKES  MORE  SENSE  IF  READ  TWICE 
(TESTS)  ADEQUATE  OVERVIEW  OF  MATERIAL 
A  LITTLE  TROUBLE,  BUT  I  DID  WELL  OVERALL 
(C2)  WITHOUT  A  DOUBT  THE  HARDEST  SECTION 


Subject  5  -  TERMS  AND  GRAPHICS 

TIME:  3.5+3=6.5/2=3.25 

PRETEST:  66+70=136/2=68 

FINAL:  90+80=  170/2=85 

LESSON  EXAM  5A:  87+83=  170/2=85 

LESSON  EXAM  5B :  72+77=  149/2=74.5 

FIXES :  NONE 

COMMENTS:  OUTSTANDING  PE 

EXCELLENT  FORMAT  FOR  QUIZZES 
(CBI)  GOOD  EXAMPLES 
SOME  REFRESHER,  SOME  NEW 
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Subject  6  -  MILITARY  BRIEFINGS 


TIME:  3.5+2.25=5.75/2=2.87 

PRETEST:  64+85=149/2=74.5 

FINAL:  80+85=165/2=82.5 

LESSON  EXAM  6A:  93+93=186/2=93 

LESSON  EXAM  6B :  73+80=153/2=76.5 

FIXES :  NONE 

COMMENTS:  (CBI)  WAS  THE  BEST  BECAUSE  IT  WAS  LAST 
I  KNEW  SOME  OF  IT,  SOME  WAS  NEW 
WAS  WELL  LAID  OUT. . .GOOD  SEQUENTIAL  ORDER 
EXCELLENT  EXPLANATION  OF  BRIEFING  TYPES 
(TESTS/QUIZZES)  LONG  ENOUGH  TO  GIVE  GOOD 
INDICATION 


Subject  7  -  STAFF  INTEGRATION 

TIME:  1.25+1.5=2.75/2=1.38 

PRETEST:  70+86=156/2=78 


FINAL:  ? 

LESSON  EXAM  7A:  92+68=160/2=80 

FIXES :  None 

COMMENTS:  (CBI)  EXCELLENT  SOURCE  OF  LEARNING 

INFO  WELL  PRESENTED,  PROGRESSED  LOGICALLY. . . 
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BATTLE  STAFF  TRAINING  SYSTEM 
SI  BETA  TESTING  SUMMARY  AND 

ACTIONS  REQUIRED  19  Dec  94 


I.  Demographic  Data:  Student  officers  are  ARNG  and  AC  officers 
serving  with  the  ARNG  on  active  duty  or  preparing  to  attend  lOAC. 
Officers  have  completed  lOBC  and  served  as  platoon  leaders, 
assistant  staff  officers  at  battalion  and  brigade  with  one 
currently  serving  as  a  signal  officer.  One  officer  has  completed 
CAS3.  All  are  college  graduates.  Average  time  on  active  duty 
was  10  years.  One  of  the  officers  had  two  rotations  to  the  NTC. 
General  comments  from  all  officers  were  neutral  to  negative  to 
traditional  correspondence  courses  and,  to  a  lesser  extent, 
resident  training.  Comments  to  the  quality  of  Army  publications 
was,  overall,  negative. 


II.  Post  Course  Survey  Results; 

Of  the  12  questions  on  the  post  course  survey,  67%  were 
positive  (+)  on  the  course  of  instruction  with  8%  neutral  (0) ; 
and  24%  negative  comments.  (For  positively  based  questions  a  + 
was  indicated  by  a  5  or  4;  for  negatively  phrased  questions  a  + 
was  indicated  by  marking  a  1  or  2;  3  was  the  neutral  range) . 

Overall  Comments: 

Good  condensed  version  of  practical  knowledge. 

Most  valuable  component  was  (CBI)  PEs  and  job  aids. 

Suggestions : 

More  CBI  and  PEs . 


Time  to  complete:  TEXT:  22.0 

CBI :  7.5 

TOTAL  TIME:  29.5  HOURS 
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Siibject  1  -  SI  Organization  and  Planning 

TIME :  TEXT :  lA  4 , 0  IB  2.0 

CBI:  lA  .5  IB  1.75 

PRETEST:  60% 

FINAL:  89% 

LESSON  EXAM  lA:  83% 

LESSON  EXAM  IB:  91% 

FIXES:  ALL  LESSON  EXAMS:  ERROR  22336;  70-80%  FEEDBACK 

STATEMENT;  REPLACE  MARGINAL/MARGINAL  FILE  NAME 

LSN  lA:  MORE  TIME  ON  FEEDBACK  PAUSE 

LSN  IB:  PE  -  FIX  TF  ORG  DIAGRAM  7  VS  6  COS 

PE2:  SAYS  IT  IS  PE  IB  PT  II 

TEXT??  BDE  OPORD  ANNEX  Q  =  ANNEX  0 

FINAL  EXAM  -  CAS  MGT,  ?  CT  OR  FT 


COMMENTS:  NONE 


Subject  2  -  STRENGTH  MANAGEMENT 


TIME: 

TEXT: 

2A  1.0 

2B  .75 

2C 

2.25 

CBI : 

2A  .25 

2B  .25 

2C 

1.75 

PRETEST:  44% 

FINAL: 

94% 

LESSON 

EXAM  2A: 

90% 

LESSON 

EXAM  2B: 

100% 

LESSON 

EXAM  2C: 

90% 

FIXES : 

EXAM 

MARGINAL 

=  MARGINAL 

PE  PT 

I  -  Can' 

t  click  on 

categories 

Forcesthat  -  needs  space 
PE  PT  I  -  check  answers  Q3 , 4 
PE  PTII  -  check  answers/spelling  Q2,3 
PE  PT  IV-  check  QIO,  answer  A 

COMMENTS:  None 
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Subject  3  -  REPLACEMENT  OPERATIONS 

TIME:  TEXT:  3A  2 , 0  3B  1.00  3C  1.00 

CBI:  3A  .25  3B  .25  3C  1.50 

PRETEST:  46% 

FINAL:  93% 

LESSON  EXAM  3A:  100% 

LESSON  EXAM  3B:  86% 

LESSON  EXAM  3C:  89% 

FIXES:  TEXT  -  NO  PE  3A 

CBI  QUIZ  MISSING? 

MARGINAL  =  MARGINAL 

MENU  -  REPL  ORIENTATION  SUPPLY  ?  WORDING 
3B  TEST  -  COMPANY  ORIENTATION  . . .  CHECK  ANSWER 
CHECK  QUIZ  3 Cl 
CHECK  QUIZ  3C2 


COMMENTS:  NONE 


Subject  4-  MEDICAL  OPERATIONS 

TIME:  TEXT:  4A  1 . 5  4B  1.00  4C  1.25 

CBI:  4A  .25  4B  .25  4C  .25 


PRETEST:  75% 

FINAL:  97% 

LESSON  EXAM  4A:  88% 

LESSON  EXAM  4B:  90% 

LESSON  EXAM  4C:  80% 


FIXES:  4A  INTRO  RED  LETTERS  ON  BLUE  HARD  TO  READ 

4A  TEST:  Check  Ql,  med  reg  is...,  last  Q 

4B  CBI  -  Red  on  blue  hard  to  read 

Video 

4C  Casualty  reports  flash  by  too  fast 


COMMENTS:  None 
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Subject  5  -  EPW/CIVILIAN  INTERNEES 

TIME:  TEXT:  5 A  ,75 

CBI:  5A  .25 

PRETEST:  40% 

FINAL:  90% 

LESSON  EXAM  5A:  100% 

FIXES:  Check  lesson  exam  #questions/coraputations 

COMMENTS:  None 

Subject  6  -  OTHER  PERSONNEL  SERVICE  SUPPORT 

TIME:  TEXT:  6A  1 . 0  6B  1.00 

CBI:  6A  .25  6B  .25 

PRETEST:  27% 

FINAL:  91% 

LESSON  EXAM  6A:  92% 

LESSON  EXAM  6B:  100% 

FIXES:  6B  TEXT,  CHECK  PG  3,  PROMOTIONS  ... CPT . FOR . CLARITY 

6B  FINAL  EXAM,  POW  QUESTION  -  FEEDBACK?? 

COMMENTS:  None 

Subject  7  -  MORALE,  WELFARE,  AND  RECREATION 

TIME:  TEXT:  7A  .50  7B  1.00  7C  .5  7D  .50 

CBI:  7A  .25  7B  .50  7C  .25  7D  .25 

PRETEST:  62% 

FINAL:  94% 

LESSON  EXAM  7A:  83% 

LESSON  EXAM  7B :  85% 

LESSON  EXAM  7C:  84% 

LESSON  EXAM  7D:  89% 

FIXES;  Check  exams  path,  postal  is  in  7B,  C,  D 
COMMENTS:  None 
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BATTLE  STAFF  TRAINING  SYSTEM 
CHAPLAIN  BETA  TESTING  SUMMARY  AND 
ACTIONS  REQUIRED  23  Dec  94 


I.  Demographic  Data:  Student  officer  was  a  LTC  USAR  officer 
serving  as  a  Division  Staff  Chaplain  with  the  70th  Div  (IT)  on 
active  duty  at  Ft  Banning.  Officer  has  completed  OBC,  OAC,  and 
CGSC.  Age  was  46,  education  level  was  college  graduate  with 
masters  in  divinity.  Time  on  active  duty  was  20  years  with  2 
years  in  a  500  bed  hospital,  9  years  as  a  battalion  chaplain,  3 
years  as  a  brigade  chaplain,  and  6  years  as  the  deputy  and 
division  chaplain.  Officer  did  not  have  experience  at  the  NTC. 
General  comments  from  the  officer  was  negative  to  traditional 
correspondence  courses,  negative  to  reserve  component  resident 
courses,  and,  positive  to  resident  training.  Officer  uses  a 
computer  in  his  work  and  considers  himself  relatively  at  ease  in 
using  a  computer.  His  preference  for  instruction  is  first,  SGI, 
followed  by  CBI . 


II.  Post  Course  Survey  Results: 

Of  the  12  questions  on  the  post  course  survey,  75%  were 
positive  (+)  on  the  course  of  instruction  with  8%  neutral  (0) ; 
and  17%  negative  comments.  (Note:  the  Chaplain's  course  has 
very  limited  CBI  -  introductions  and  tests  only;  no  exercises) . 
(For  positively  based  questions  a  +  was  indicated  by  a  5  or  4 ; 
for  negatively  phrased  questions  a  +  was  indicated  by  marking  a  1 
or  2;  3  was  the  neutral  range) . 

Overall  Comments: 

Good,  solid  course  for  entry  level  chaplains.  It  contains 
all  the  necessary  basics  they'll  need  for  field 
operations  at  the  TOC. 

Quizzes  with  immediate  feedback  to  right  and  wrong  answers 
was  a  good  . tool . 

Suggestions: 

Integrate  some  additional  text  and/or  (CBI)  exercises  into 
the  program  to  enhance  the  readings  in  the  text. 
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Sxibject  1  -  RELIGIOUS  SUPPORT  ACTIVITIES 
TIME :  ? 

PRETEST:  83% 

FINAL:  92% 

LESSON  EXAM  lA:  90% 

LESSON  EXAM  IB:  95% 

FIXES:  lA  Pretest  -  feedback  on  missed  questions  - 

error,  variable  missing,  22302. 

IB  Possible  question  around  suicide,  what  do  you 
do  with  a  soldier  who  in  "confidence"  tells 
you  about  suicidal  ideation?? 

Final  Exam:  Giving  same  answers  on  post  test  as 

pretest  I  didn't  get  same  questions  as  wrong. 
NOTE:  check  all  questions  -  pre  and  post 

test . 

Check  question  on  murder  of  a  civilian  - 
correct?? 


COMMENTS : 
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Subject  2  -  STAFF  FUNCTIONS 


TIME: 

PRETEST:  100% 

FINAL:  92% 

LESSON  EXAM  2A:  100% 

LESSON  EXAM  2B:  100% 

FIXES : 

Final  -  check  question  on  advising  cdr  on  religious 

holidays,  may  be  incorrect  -  Ramadan,  in  SW  Asia. 

Question  about  soldier  killing  a  buddhist  monk 

is  in  conflict  with  question  about  soldier 
committing  murder  in  Lesson  I. 


COMMENTS:  2A  Good  test,  very  helpful.  I  liked  the 

matching  questions. 

Subject  2,  2a,  pgl:  Para  2b:  Pa(s)toral 
their 

Discussion  of  CP,  AMCP  could  use  a  diagram 

Lesson  2A  reads  like  a  phone  book  -  Lesson  1 
has  better  movement  and  easier  to  read. 

Lsn  2B,  pg  6,  top  para;  "immorality 
counseling"  rephrase  it??  Sounds  like  we're 
teaching  now  to  be  immoral . 

Quotes  early  in  text  are  course,  could  turn 
off  young  chaplain. 


D-12 


BATTLE  STAFF  TRAINING  SYSTEM 
S2  BETA  TESTING  SUMMARY  AND 
ACTIONS  REQUIRED  28  November  1994 


I.  Demographic  Data:  Student  officers  are  ARNG  officers  serving 
as  Bradley  platoon  leaders  in  the  ARNG.  Officers  completed  lOBC 
one  week  prior  to  start  of  testing.  Average  age  was  24, 
education  level  was  college  graduate  with  one  graduate  student. 
Average  time  on  active  duty  (except  for  training)  was  0  with 
average  reserve  duty  time  3.25  years.  None  of  the  officers  had 
experience  at  the  NTC.  General  comments  from  all  officers  were 
negative  to  traditional  correspondence  courses  and,  to  a  lesser 
extent,  resident  training.  Comments  to  the  quality  of  Army 
publications  was,  overall,  negative. 


II.  Post  Course  Survey  Results: 

Of  the  15  questions  on  the  post  course  survey,  93%  were 
positive  (+)  on  the  course  of  instruction  with  7%  neutral  (0) ; 
there  were  no  negative  comments.  (For  positively  based  questions 
a  +  was  indicated  by  a  5  or  4;  for  negatively  phrased  questions  a 
+  was  indicated  by  marking  a  1  or  2 ;  3  was  the  neutral  range) . 

Overall  Comments: 

Course  was  a  good  supplement  to  the  FMs 

Very  good  and  challenging 

Suggestions : 

More  real  world  examples  and  (CBI)  exercises 

More  computer  instruction. 
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Subject  1  -  IPB 

TIME:  17+10.5=27.5/2=13.75 

PRETEST:  44+65=109/2=54.5 

FINAL:  68+92=160/2=80 

LESSON  EXAM  lA:  78+92=170/2=85 
LESSON  EXAM  IB:  80+72=152/2=76 
LESSON  EXAM  1C:  40+55=95/2=47.5 

LESSON  EXAM  ID:  80+46=126/2=63 
LESSON  EXAM  IE:  61+92=153/2=76.5 

FIXES:  Feedback  on  all  questions  _  ^ 

Threat  Eval,  question  4  (composition) ,  and  14 
(strength)  conflict  w/book? 

Tutorial  on  DocTemp  needs  more  on  how  to  use, 
gather. . . 

COMMENTS:  METT-T  called  M-TETT  by  USAIS 
Good  instruction 

Longer  quizzes/more  quizzes  (consistent) 

To  much  material 
Good  exercises 

Need  ability  to  print  out  maps,  charts... 
Held  my  interest  well 
More  PEs  (CBI) 

Bigger  table  of  contents 

Computer  is  excellent  learning  tool 

Need  practical  FM's  like  the  student  guide 

Excellent  visual  displays 
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Subject  2  -  R&S 

TIME:  7+7,25=14.25/2=7.12 

PRETEST:  48+75=123/2=61.5 


FINAL:  88+84=172/2=86 


LESSON  EXAM  2A: 
LESSON  EXAM  2B : 
LESSON  EXAM  2C 
LESSON  EXAM  2D: 


62+85=147/2=73 . 5 
37+75=112/2=56 
42+71=113/2=55.5 
88+62=150/2=75 


FIXES:  2C  QUIZ  gave  incorrect  feedback?? 


COMMENTS:  Text  used  very  well  (multiple) 

Didn't  do  well  on  PE 
Reading  was  valuable 
Best  yet 

Best  instruction  and  easy  to  comprehend 
(Tests ..) Good  basis  of  test  material 


Subject  3  -  Collection  Plan 
TIME:  3+2=5/2=2,5 

PRETEST:  55+70=125/2=62,5 


FINAL:  70+90=160/2=80 

LESSON  EXAM  3A:  14+86=100/2=50! 

LESSON  EXAM  3B :  55+90=145/2=72.5 

FIXES:  Emmii  -  ease  of  use/dependability 

(multiple) 

COMMENTS:  I  looks  up  what  I  missed  and  found  info  was 
there 

I  read  this  and  it  stuck  in  my  head 

Tests  -  a  lot  better 

Emmii  -  bugs  need  to  be  fixed 
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Subject  4  -  OPSEC 

TIME:  3 . 5+4 . 5=8/2=4 

PRETEST:  42+66=108/2=54 

FINAL:  61+12=129/2=69 .S 

LESSON  EXAM  4A:  33+83=116/2=58 

LESSON  EXAM  4B:  67+70=137/2=68.5 

FIXES:  4A  Threat:  1st  exam,  ans  A,  computer  said  NO  -  A 

COMMENTS:  Should  have  bigger  quizzes  and  more  PEs  (CBI) 
£^06 1 1 ©nt 

short  and  concise,  a  lot  of  material 


Subject  5  -  INTELLIGENCE  ESTIMATE 

TIME:  2.25+2=4.25/2=2.12 

PRETEST:  67+67=134/2=67 

FINAL:  94+88=182/2=91 

LESSON  EXAM  5A:  40+67=107/2=53.5 

LESSON  EXAM  5B :  23+100=123/2=61.5 

FIXES :  None 

COMMENTS:  Reading  was  valuable 

(CBI)  Still  best  source  of  learning 
Very  difficult  quiz  _  . 

(CBI)  was  more  in-depth,  good  covering  points 
Good  use  of  text  with  CBI 
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Subject  6  “  S2  OPERATIONS 

TIME:  5.75+2.75=8.5/2=4.25 

PRETEST:  22+27=49/2=24.5 

FINAL:  74+83=157/2=77.5 

LESSON  EXAM  6A:  44+67=111/2=55.5 

LESSON  EXAM  6B :  40+60=100/2=50 

FIXES :  None 

COMMENTS:  I  am  getting  the  hang  of  grasping  knowledge 

(CBI)  Computer  is  where  I  got  all  my  knowledge 
Did  a  good  job  -  needs  more  PEs  {CBI) 

Quizzes  helped  me  prepare  for  the  final 


Subject  7  -  BN  HQ  COORDINATION 


TIME:  1.5+2.5=4/2=2 

PRETEST:  70+70=140/2=70 

FINAL:  95+80=175/2=87.5 

LESSON  EXAM  7A:  81+44=125/2=62.5 

LESSON  EXAM  7B :  74+56=130/2=65 

FIXES:  7B  Last  question  on  quiz  didn't  include  correct 

answer 

COMMENTS:  Gave  good,  simple  explanation 
Mostly  text  based  -n  more  CBI 
Good  reading  and  very  valuable  tests 
Combination  of  all  quizzes  helped  me  prepare  for 
final 
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BATTLE  STAFF  TRAINING  SYSTEM 
S3  BETA  TESTING  SUMMARY  AND 
ACTIONS  REQUIRED  29  November  1994  (Final) 


I.  Demographic  Data:  Student  officers  are  AC  officers  that  ha 
comnleted  or  awaiting  the  advanced  course.  Of ficers  completed 
lOBC  have  served  as  platoon  leaders,  company  executive  officers 
and  ;ss!stan?  battalion  or  brigade  staff  officers.  Average  age 

was  28.5,  education  level  was  college  graduate^  tim^l^S  vllls 
active  duty  5.5  years  with  average  reserve  duty  time  1.5  years. 
One  officer  had  two  rotations  at  the  NTC.  General  comments  from 
all  officers  were  positive  to  the  small  group _ instruction  (SG  ) 
instructional  methodology,  negative  to  traditional 

courses  and,  to  a  lesser  extent,  resident  training.  Comments  to 
the  Quality  of  Army  publications  was,  overall,  negative. 

Officers  were,  generally,  computer  literate  with  one  officer 
highly  knowledgeable. 


II.  Post  Course  Survey  Results: 

Of  the  12  questions  on  the  post  course _ survey ,  83%  . 

positive  (+)  on  the  course  of  instruction  with  S-s  neutral  (0)  and 
8%  negative  (one  student  preferred  the  SGI  method  of  instruction 
then  CBI  over  all  other  forms.  (For  positively  based  questions  a 
+  was  indicated  by  a  5  or  4;  for  negatively  phrased  questions  a  + 
was  indicated  by  marking  a  1  or  2;  3  was  the  neutral  range) . 


Overall  Comments: 

FM  7-20  (CH  2  &  7)  excellent  references. 

Reading  material:  sometimes  too  much. 

PEs  (CBI)  an  excellent  tool.  Should  incorporate 
more  PEs . 

Matching  were  incorrect  (fixed) . 

Excellent  system  to  serve  as  an  educational  tool  tor 
the  battalion  battle  staff. 


Suggestions : 

At  USAIS:  METT-T  =  M-TETT 
OCOKA  =  OAKOC 

Scheme  of  Maneuver  =  Maneuver 
Scheme  of  Fire  Support  =  fires 
Next  to  last  question  -  lesson  exam  6b;  blank  screen 
shows  no  question  but  has  4  answers. 
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Subject  1  -  Synchronization 
TIME:  2  hours 

Text:  1.25  CBI :  .75 

PRETEST:  60% 

FINAL:  87% 

LESSON  EXAM:  84% 

FIXES:  Matching  questions  (done) . 

COMMENTS:  Syst era/program  is  an  excellent  tool. 

I  think  SGI  concept  is  best 
This  system  is  best  after  SGI . 

It  will  provide  a  great  educational  tool  for 
soldiers . 


Subj  ect  2  -  TF  CSS 

TIME :  5.5  hours 

Text:  4.45  CBI:  .75 

PRETEST:  60% 

FINAL:  90% 

LESSON  EXAM  2A:  70% 

LESSON  EXAM  2B :  95% 

LESSON  EXAM  2C:  93% 

LESSON  EXAM  2D:  95% 

FIXES:  Matching  (fixed) . 

2D  exam  computed  %  correct  INCORRECTLY. 

Check  pre-test;  post-test  and  2D  lesson  exam. 

COMMENTS:  A  lot  of  information  to  learn. 

Matching  incorrect  (fixed) . 

Material  clear  and  concise. 

Student  guide  provided  concise  information. 
Computer  based  supported  performance  objectives. 
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Subject  3  -  METT-T  Analysis 
TIME :  3.1  hours 

Text:  1.9  hours  CBI:  1.25  hours 

PRETEST:  70% 


FINAL:  90% 

LESSON  EXAM  3A:  88% 

LESSON  EXAM  3B:  70% 

FIXES:  -  PEs  3B1  and  3B2  did  not  follow  course  map 


(fixed) . 


COMMENTS:  CBl/program  provided  excellent  examples  for 

subject  matter  ...  clear  and  concise.  PE  3 
good  exercise.  Material  was  easy  to 
understand. 


Subject  4-  Courses  of  Action 
TIME :  2,5  hours 

Text:  1.75  CBI:  .75 

PRETEST:  75% 

FINAL:  95% 

LESSON  EXAM  4A:  67% 

LESSON  EXAM  4B :  100% 

FIXES :  None 

COMMENTS:  Reading  material  for  lesson  was  adequate. 

Text  was  simple  and  easy  to  understand  . . . 
Good  PE  ...  opportunity  to  actually  analyze 
different  COA. 

(CBI)  Graphic  representation  really  helps  out 
a  picture  is  worth  a  thousand  words . 
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Subject  5  -  Orders  Process 
TIME:  1.1  hours 

Text;  .95  CBI :  .25 

PRETEST;  85% 

FINAL:  100% 

LESSON  EXAM  5A;  100% 

FIXES:  Matching  all  incorrect  (fixed). 


COMMENTS:  (In  OPORD)  Current 
of  maneuver  as  maneuver 
fires. 


USAIS  teaching  refers  to  scheme 
and  scheme  of  fire  support  as 


Subject  6  ”  Supervise  execution 

TIME;  2.25  hours 

Text:  1.75  CBI:  .5 

PRETEST:  90% 

FINAL:  100% 

LESSON  EXAM  6A:  87% 

LESSON  EXAM  6B :  93% 

FIXES:  6B  exam:  Blank  screen,  no  question;  however,  4 

responses:  feeding  location,  maneuverability, 
survivability,  and  mobility. 

One  question  had  two  of  the  same  choices:  Q:  The 

minimum  information  display  posted  on  the  O&I  map 
includes:  ...  personnel  status  is  listed  twice. 

COMMENTS:  Clear,  simple  and  easy  to  understand. 


D-21 


BATTLE  STAFF  TRAINING  SYSTEM 
S3Air  BETA  TESTING  SUMMARY  AND 

ACTIONS  REQUIRED  30  Jan  95 


I.  Demographic  Data:  Student  officers  are  AC 

assignment  to  the  Infantry  Officers  Advanced  Course  (lOACK  The 
Infantry  Officer  has  a  BS  having  served  as  an  S3  LNO,  Ritie 
Platoon  Leader,  AT  Platoon  Leader  in  an  Airborne  Regiment  and 
having  completed  lOBC,  Airborne,  Ranger,  and  Jumpmaster.  Officer 
hid  ole  rotation  at  JRTC.  The  ADA  Officer  has  a  BA  having  served 
in  an  ADA  brigade  with  no  Battles  Staff  experience.  Officer 
completed  ADA  OBC.  Average  age  was  30,  education  level  was 
college  graduate.  Average  time  on  active  duty  (except  for 
training)  was  7  years  with  average  reserve  duty  time  2  years. 

NoL  of  the  officers  had  experience  at  the  NTC.  General  comments 
from  all  officers  were  negative  to  traditional  correspondence 
courses  and,  to  a  lesser  extent,  resident  training.  Comments  to 
the  quality  of  Army  publications  was,  overall,  negative. 


II.  Post  Course  Survey  Results: 

Of  the  12  questions  on  the  post  course  survey,  75-s  were 
positive  (+)  on  the  course  of  instruction  with  8%  neutral;  there 
were  17%  negative  comments.  (For  positively  based  questions  a  + 
was  indicated  by  a  5  or  4 ;  for  negatively  phrased  questions  a  + 
was  indicated  by  marking  a  1  or  2;  3  was  the  neutral  range) . 

Overall  Comments: 

A  lot  of  good  information  in  a  very  condensed  block  of 

instruction. 

Suggestions:  Suggest  this  training  be  implemented  in  the 

National  Guard  and  the  Active  Component. 

The  best  implementation  of  the  training  is  to  present 
the  training  before  the  officer  occupies  a  staff 
position. 

Time  to  Complete:  TEXT:  51.0 

CBI:  1.75 

TOTAL  TIME:  52.75 
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Subject  1  “  AIR  SYSTEMS 

TIME:  TEXT:  lA  14.0 

CBI:  lA  .25 

PRETEST:  60% 

FINAL:  90% 

LESSON  EXAM  lA:  80% 

LESSON  EXAM  IB:  90% 


IB  7.0 
IB  .25 


FIXES :  NONE 


COMMENTS:  The  instruction  provided  in  the  BSTS  material  is 
excellent  . 


S\ibject  2  -  AIR  OPERATIONS 


TIME:  TEXT: 

CBI: 


2A  12.5  2B  10.5 
2A  .25  2B  .5 


2C  3 . 0  2D  4 . 0 
2C  .25  2D  .25 


PRETEST:  62% 

FINAL:  93% 

LESSON  EXAM  2A:  70% 

LESSON  EXAM  2B :  80% 

LESSON  EXAM  2C:  80% 

LESSON  EXAM  2D:  80% 

FIXES:  2B  did  not  give  thumbs  up  in  main  menu. 

2B  intro  lengthy  but  overall  pretty  good. 

2C  Text:  Ref  to  FM  1-100  no  CH4  or  APP  F,  G,  H 


COMMENTS:  Did  not  have  FM  100-42 
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BATTLE  STAFF  TRAINING  SYSTEM 
S4  BETA  TESTING  SUMMARY  AND 
ACTIONS  REQUIRED  3  December  1994 


I.  Demographic  Data:  Student  officers  are 

tat  jrf'JdLa^iofiSverwaroSllege  graduate.  Average  time  on 
:Sti;^d;tf  5.S  years  with  average  rese^e  ^-ty  trme  1^5  years^ 

S?  ™  ?L“:au"groSr?Struction  (SOI) 

rrnalitv  of  Armv  publications  was,  overall,  negative. 
nf?lS?s  wLr  qlnLally,  computer  literate  with  one  officer 
SigM^kho:^edgXS?e.  One  officer  served  as  a  support  platoon 
leader/assistant  S4 . 

II,  Post  Course  Survey  Results: 

Of  the  12  objective  questions  on  the  post  n 

were  positive  {  +  )  on  the  course  of  instruction  with  _ 

(0)-  there  were  two  negative  comments  that  placed  a  preference 
thi'sS  Sst?uctional  Lthodology  over  the 

methodology.  (For  positively  based  questions  a  +  was  indicated 
by  a  5  or^4;  for  negatively  phrased  questions  a  +  was  indica 
by  marking  a  1  or  2;  3  was  the  neutral  range) 

Overall  Comments: 

Reading  material  was  relatively  easy  and  focused 
specifically  on  the  performance  objectives  as  specified 
student  guide. 

overall,  this  was  an  excellent  course  to  provide  information 
and  practical  experience  to  a  Task  Force  S4  (new  or  old)  ; 
and  ?eference  material  provide  sufficient  information  which  wil 
help  a  TF  S4  understand  his  duties  and  responsibiii 

Practical  exercises  on  lessons  3B-D  were  good  exercises. 

Total  Time:  Text:  10.5  hrs  CBI  4.5  hrs  Total:  15.1  hrs 

Suggestions:  , 

Error  message  displayed  on  exams  lA,  3C,  3D,  an 

"error  displaying  file  C:\bsts\S43\Text\exam. 

2  2  3  3  G  " 

Lesson  IB  and  2B  in  student  guide;  reference  is  FM  71-2, 

CH  6  ....  should  be  CH  7 . 
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Subject  1  -  Task  Force  Resupply  Operations 
TIME: 

Text:  lA  .75  IB  1.0 

CBI :  lA  .75  IB  .25 

PRETEST:  90% 

FINAL:  100% 

LESSON  EXAM  lA:  100% 

LESSON  EXAM  IB:  100% 

FIXES:  Student  guide  ref  FM  71-2,  CH  6;  should  be  CH  7 

IB  lesson  exam  received  a  score  of  267-s 
Exam  menu  stated  3  questions  -  were  in  fact  8. 

COMMENTS:  Reading  material  easy  to  understand 

Reading  material  focused  on  objectives. 

Subject  2  -  Organization  for  Tactical  Sustainment 
TIME: 

TEXT :  2A  -  1 . 0  2B  -  1 . 0 

CBI:  2A  -  .25  2B  -  .5 

PRETEST:  64% 

FINAL:  100% 

LESSON  EXAM  2A:  90% 


LESSON  EXAM  2B:  93% 


FIXES: 


2A  Pretest,  typo  in  Q:  "  The  company  trains  . 
2B  Student  guide:  Ref  to  FM  71-2,  CH6  should 
CH  7 


COMMENTS:  Performance  measures  clear,  student  guide  materia 
easy  to  follow  which  made  understanding  of  the  material  more 
simplified. 
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STibject  3  -  Tactical  Sustainment  Operations 


TIME: 


TEXT 


CBI 


3A 

1.0 

3B 

1.0 

3C 

1.25 

3D 

1.0 

.5 
.  75 

.  5 
.  5 


PRETEST 

:  65% 

FINAL: 

94% 

LESSON 

EXAM  3A: 

83% 

LESSON 

EXAM  3B: 

100% 

LESSON 

EXAM  3C: 

84% 

LESSON 

EXAM  3D: 

80% 

FIXES : 

Add  FM 

101-10-1/2 

3C  -  Error  message 
comments) 


3D  -  Error  message 
comments) 


as  ref  in  stu  guide,  used  in  PEs 
after  lesson  exam  (see  gen 

after  lesson  exam  (see  gen 


COMMENTS:  Reading  material  (student  guide)  clear  and  y 
understand.  ^  . 

Practical  exercises  were  very  helpful  in 

understanding  the  process  in  which  the  S4 
goes  through  while  forecasting  for  the  Bn 
Task  Force. 


Subject  4- 

TIME: 


TEXT 

4A  1.1 
4B  .75 
4C  1.0 

PRETEST:  71% 

FINAL:  100% 

LESSON  EXAM  4A:  91% 

LESSON  EXAM  4B:  100% 

LESSON  EXAM  4C:  80% 


CBI 

.5 

.15 

.4 
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FIXES: 

4B  -  Exam,  error  message  after  lesson  exam  . . . . 
COMMENTS:  Good  exercises 
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BATTLE  STAFF  TRAINING  SYSTEM 
FSO  BETA  TESTING  SUMMARY  AND 
ACTIONS  REQUIRED  2  February  1995 


I.  Demographic  Data:  Student  officers  are  AC 

assignment  to  the  Infantry  Officers  advanced  Course  (lOACK 

OBC  Averacre  age  was  30,  education  level  was  co^ege  graduate. 
AveragffSfon^ctlve  duty,  (except  for  training)  wae  7  years 
with  average  reserve  duty  time  2  years  None  ^ 

experience  at  the  NTC.  General  comments 

negative  to  traditional  correspondence  courses  and,  to  a  less 
SSent  resident  training.  Comments  to  the  quality  of  Army 
publications  was,  overall,  negative. 

I,  Post  Course  Survey  Results: 

Of  the  12  questions  on  the  post  course _ survey,  92%  were 
r-naitive  (  +  )  on  the  course  of  instruction  with  6s  neutral  (0)  , 
there  were  no  negative  comments.  (For  positively  based  questions 
a  f wafindiL^ed  by  a  5  or  4;  for  negatively  phrased  questions  a 
+  was  indicated  by  marking  a  1  or  2;  3  was  the  neutral  range)  . 

Overall  Comments : 

Neat  stuff 


Suggestions : 

Do  more  CBI  and  more  courses  (lOBC,  CAS3) 


Time  to  Complete :  TEXT 

CBI 

TOTAL  TIME 


56 . 90 
6 . 05 
62 . 95 
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Subject  1  -  Fire  Support  Planning  and  Coordination 


TIME: 


TEXT : lA  6 . 2  IB  8.4 
CBI:1A  ,251B  1.6 


1C  12.0 
1C  .75 


ID 

ID 


8.3 

.75 


PRETEST:  56.5% 

FINAL:  91% 

LESSON  EXAM  lA: 
LESSON  EXAM  IB: 
LESSON  EXAM  1C: 
LESSON  EXAM  ID: 


80% 

81% 

88% 

80% 


FIXES:  Check  matching  in  IB 

Check  NEXT  buttons  in  IB 
Check  crrids  in  PE  IB 

Seek  positioning  of  feedback  screens  over  answer  in 
ID 


COMMENTS :  none 


Subject  2  -  Targeting 

TIME:  TEXT:  11.0 

CBI :  . 6 

PRETEST:  76% 

FINAL:  84% 

LESSON  EXAM  2A:  88% 

FIXES:  Check  lesson  exam  feedback  screen  size  and  next 

button 

COMMENTS :  none 
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Subject  3  -  Fire  Support  Assets 

TIME:  TEXT:  11.0 

CBI :  2.1 

PRETEST:  94% 

FINAL:  95% 

LESSON  EXAM  3A:  91% 

FIXES:  Check  video  for  operation 

COMMENTS :  none 
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BATTLE  STAFF  TRAINING  SYSTEM 
ADAO  BETA  TESTING  SUMMARY  AND 

ACTIONS  REQUIRED  9  Jan  95 


I.  Demographic  Data:  Student  officers  pY^^^The 

assignment  to  the  Infantry  Officers  Advanced  bourse  (lOACK 
Infantry  Officer  has  a  BS  having  served  as  an  S3  LNO,  RiOe 
Platoon^LeaLr,  AT  Platoon  Leader  in  an  Airborne  Regiment  and 
havina  completed  lOBC,  Airborne,  Ranger,  and  Jumpmaster.  Officer 
hS  SL  ?o?aticn  at  J^TC.  The  ADA  Officer  has  a  BA  having  served 
in  an  ADA  brigade  with  no  Battles  Staff  experience, 
completed  ADA  OBC.  Average  age  was  30, 

college  graduate.  Average  time  on  active  duty  5ears 

trainincf)  was  7  years  with  average  reserve  duty  time  2  years. 
Non^of  the  officers  had  experience  at  the  NTC.  General  comments 
from  all  officers  were  negative  to  traditional  corresponden 
coSrsei  and,  to  a  lesser  extent,  resident  framing  Comments  to 
the  quality  of  Army  publications  was,  overall,  negative. 


II.  Post  Course  Survey  Results: 

Of  the  12  questions  on  the  post  course  survey,  83%  were 
Dositive  (+)  on  the  course  of  instruction  with  0-i  neutral  (  , 

there  were  17%  negative  comments.  (For  positively  based 
gueS?ions  a  +  was  indicated  by  a  5  or  4 ;  for  negatively  phrased 
questions  a  +  was  indicated  by  marking  a  1  or  2;  3  was  the 
neutral  range) . 


Overall  Comments: 

Very  professional  course;  knowledge  contained  is  very  up  to 

Most  valuable  component  is  the  student  guide;  job  aids 
outstanding. 


Suggestions:  Suggest  this  training  be  implemented  in  the 

National  Guard  and  the  Active  Component. 

The  best  implementation  of  the  training  is  to  presen 
the  training  before  the  officer  occupies  a  staff 
position. 


Time  to  Complete:  TEXT: 

CBI : 


17 . 2 
1.75 


TOTAL  TIME:  18.95 
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STibject  1  -  AIR  DEFENSE  PLANNING 

TIME:  TEXT:  lA  4.25 

CBI:  lA  .25 


IB  3.45 
IB  .25 


1C  2.5  ID  2.0 

1C  .5  ID  .25 


PRETEST:  75% 

FINAL:  88% 


LESSON  EXAM  lA 
LESSON  EXAM  IB 
LESSON  EXAM  1C 
LESSON  EXAM  ID 


95% 

85% 

88% 

83% 


FIXES:  CBI  Gave  credit  to  staff  integration  when  AD 

employment  was  completed. 


COMMENTS : 

sections 

provided 


The  tactical  decision  making  and  staff  integration 
are  critical  to  the  new  ADAO.  The  instruction 
in  the  BSTS  material  is  excellent: 


Subject  2  -  AIR  OPERATIONS 

TIME:  TEXT:  2A  3 . 0 

CBI:  2A  .25 

PRETEST:  76% 

FINAL:  90% 


2B  2.0 
2B  .25 


LESSON  EXAM  2A:  93% 


LESSON  EXAM  2B :  90% 

FIXES:  None,  is  in  accordance  with  current  ADA  doctrine. 

COMMENTS:  You  can  reference  the  Visual  Aircraft  Recognition 
(VACR)  on  CD-ROM  that  the  Army  currently  issues. 
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battle  staff  training  system 

CHEMO  BETA  TESTING  SUMMARY  AND 

ACTIONS  REQUIRED  25  Jan  95 


I  Demographic  Data:  Student  officers  are  AC 

assignment  to  the  Infantry  Officers  Advanced  Course  (lOACK 

£  S:r  sficer 

college  graduate.  Average  time  on  active  duty 

training)  was  7  years  with  average  reserve  duty  ^ jear^^^^^^ 

??Sm  an^officerfwLe^LgS^iv^^r traditional  ‘  correspondence 
crsS^anaf^n  TssLr  Ltent,  resident  training^  Comments  to 
the  quality  of  Army  publications  was,  overall,  negati 


II.  Post  Course  Survey  Results: 

Of  the  13  questions  on  the  post  course _ survey ,  62%  were 
positive  (+)  on  the  course  of  instruction  with  8%  neutral;  and 
Negative  comments.  (For  positively  based  questions  a  +  was 
indicated  by  a  5  or  4;  for  negatively  phrased  questions  a  +  was 
indicated  by  marking  a  1  or  2 ;  3  was  the  neutral  range) . 


Overall  Comments : 

Use  of  CBI  is  admittance  of  failure  of  the  Army's  Officer 
training  program. 


Suggestions : 

Do  not  repeat  information  between  text  and  FMs. 
Ensure  test  questions  test  the  required  readings. 
Have  different  questions  on  re-tests. 


Time  to  Complete:  TEXT: 

CBI : 


43.75 

2.25 


TOTAL  TIME:  46.0 
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Subject  1  -  ORGANIZATION  AND  DUTIES 
TIME: 


TEXT: 
CBI : 


PRETEST:  41.5% 

FINAL:  92% 

LESSON  EXAM  lA: 
LESSON  EXAM  IB: 


lA  15 
lA  .  5 


IB 

IB 


9.5 


75.5% 

85.0% 


FIXES:  Spelling  error  in  IB,  TCS,  should  be  biological 

COMMENTS:  I  believe  doctrine  has  changed  to  a  NO  NBC  use. 

Delete  reading  in  FM  71-123  CH  3 ,  /• 


Subject  2  -  TF  NBC  OPERATIONS 

TIME: 


TEXT: 
CBI : 


2A  2.5 
2A  .5 


2B  3.0  2C 
2B  .25  2C 


3.5  2D 
.25  2D 


10.25 

.25 


PRETEST:  43% 

FINAL:  91% 


EXAM 

2A: 

85 

EXAM 

2B: 

75 

EXAM 

2C: 

85 

EXAM 

2D: 

97 

FIXES: 


TEXT 

CBI: 


CBI 

CBI 


:  2B  PG  4  LINE  29,  M2 6 5  SHOULD  BE  M2 5 6 
2b!  thumbs  up  on  lesson  MENU  NOT  SUBJECT 
MENU 

2B  4  OF  10  QUESTIONS  ON  TABLES 
2D ,  GOT  THUMBS  UP  ON  LESSON  MENU  BUT  NOT 

MAIN  MENU  _ 

2D,  IN  TEST  THE  WORK  WINDS  SHOULD  BE  SMOKE 


COMMENTS:  Less  text  readings,  more  CBI  and  exercises. 
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BATTLE  STAFF  TRAINING  SYSTEM 
SI GO  BETA  TESTING  SUMMARY  AND 

ACTIONS  REQUIRED  B  Jan  95 


,  .  TV— 4-^  officsirs  air©  AC  officcirs  awaiting 

I.  .Demographic  Data:  Student  off 

assignment  to  tne  inranury  Rifle 

roT?^re1rXa?^^-  ora^?ivf duty  (except 

None"o?®lhro£nce;f  hlf  NTC.  General  comments 

the  quality  of  Army  publications  was,  overall,  negative. 


II.  Post  Course  Survey  Results: 

Of  the  13  questions  on  the  post  ^1^%)  . 

oositive  (+)  on  the  course  of  instruction  with  17 »  neutral  () , 
1^5-  neaative  comments.  (For  positively  based  questions  a  + 
tfs  inAiSS?ed  S  rror  4;  for  negatively  phrased  questions  a  . 
Ill  “ScStS  by  marking  a  1  or  2;  3  was  the  neutral  range)  . 

Overall  Comments : 

Course  provides  a  good  trainer/refresher  on  SINCGARS/MSE 
operations  and  capabilities  of  the  equipment. 

CBI  was  the  best  with  the  job  aids  a  close  se 

Suggestions : 

No  changes  necessary. 


Time  to  Complete :  TEXT :  15.6 

CBI:  2.0 


TOTAL  TIME: 


17.6 
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Siobject  1 
TIME; 


SIGNAL  SUPPORT 


TEXT: 

CBI: 


PRETEST:  50% 

FINAL:  86.5% 

LESSON  EXAM  lA: 
LESSON  EXAM  IB: 


FIXES: 


NONE 


lA  1.5 
lA  .25 


80- 

85- 


IB 

IB 


3.6 

.5 


COMMENTS:  DOES  NOT  DISCUSS  5TH  TENANT  OF  ALB  -VERSATILITY 

Could  be  made  to  be  more  complex  subject. 


Subject  2  -  SINCGARS  OPERATION 

TIME:  TEXT:  2A  4 . 5 

CBI :  2A  .  5 


2B  6.25 
2B  .75 


PRETEST:  50% 

FINAL:  92% 


LESSON  EXAM  2A:  80% 

LESSON  EXAM  2B :  90% 

FIXES:  None 

COMMENTS:  Reading  and  text  identical,  eliminate  some 

reading . 


\ 
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